From lcoquelle@REDACTED  Mon Oct  1 02:34:33 2007
From: lcoquelle@REDACTED (Ludovic Coquelle)
Date: Mon, 1 Oct 2007 08:34:33 +0800
Subject: [erlang-questions] Storing tuples directly in Mnesia without
	records
In-Reply-To: <708470.76594.qm@web81102.mail.mud.yahoo.com>
References: <8209f740709300012y230c1fc6s5c672d1bf473a1a2@mail.gmail.com>
	<708470.76594.qm@web81102.mail.mud.yahoo.com>
Message-ID: <d6c7b34d0709301734t4945425dxd9880076c46b1a1e@mail.gmail.com>

would it be possible to define a wrapper record? I mean something in the
line of:
{data_wrapper, SomeKey, SomeData} ... such that SomeData can be whatever you
need.

On 10/1/07, shahzad bhatti <bhatti_shahzad@REDACTED> wrote:
>
> Though, I would like to use tuples of any arity, so that clients can store
> any tuples without defining schema. But it sounds like I will have to add
> that constraint for arity and record name. Also, is it possible for insert
> to automatically add table/schema if it does not exist. And is there any
> event that I can listen to know when tuples are added to simulate blocking
> read/remove (preferably using pattern based filtering). Thanks again.
>
> *Ulf Wiger <ulf@REDACTED>* wrote:
>
> Yes, you can use tuples directly. Unlike ets, mnesia requires
> the key to be the second element in the tuple, and the first
> element needs to be a label (record_name).
>
> Mnesia will check that the inserted data is a tuple of the
> right arity, and that the first element of the tuple is equal
> to the 'record_name' of the table (this defaults to the table
> name, but can be changed with the {record_name, N} option.
>
> Remember, records are only syntactic sugar for tagged tuples.
>
> BR,
> Ulf W
>
> 2007/9/30, shahzad bhatti :
> > I am trying to build a Linda like tuple space in Erlang and though my
> simple
> > application is using ets for now, but I would like to use Mnesia to
> store
> > tuples directly to take leverage of its replication and transactions
> > capabilities. Since, Mnesia supports multiple tables that map to
> records, is
> > it possible to use tuples directly. Also, if anyone can suggest anything
> > that will make it easier to implement tuplespace with blocking get/read
> > operations. Thanks in advance.
> >
> >
> >
> > ________________________________
> > Pinpoint customers who are looking for what you sell.
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> ------------------------------
> Luggage? GPS? Comic books?
> Check out fitting gifts for grads<http://us.rd.yahoo.com/evt=48249/*http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz>at Yahoo! Search.
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071001/45769145/attachment.htm>

From bruce@REDACTED  Mon Oct  1 04:00:54 2007
From: bruce@REDACTED (Bruce Fitzsimons)
Date: Mon, 01 Oct 2007 14:00:54 +1200
Subject: [erlang-questions] Decoding of ASN.1 BER files (now with added
 UTF8)
In-Reply-To: <d3520a2f0709300626g4762ba95ld5fd632145861a9f@mail.gmail.com>
References: <46FE2280.7090603@Fitzsimons.org>
	<d3520a2f0709300626g4762ba95ld5fd632145861a9f@mail.gmail.com>
Message-ID: <470054D6.9070906@fitzsimons.org>

Anders Nygren wrote:
> On 9/29/07, Bruce Fitzsimons <Bruce@REDACTED> wrote:
>   
<snip>
>> almost work perfectly (proven for BER at least), you only need to alter
>> the <toplevel>:decode/2 to return the remaining bytes and length from
>>  <snip>
> Hi
> It seems like the asn1 compiler option does most of what You want.
> From the docs
> "undec_rest
> A buffer that holds a message, beeing decoded may also have some
> following bytes.
> Now it is possible to get those following bytes returned together with
> the decoded value.
> If an asn1 spec is compiled with this option a tuple {ok,Value,Rest}
> is returned.
> Rest may be a list or a binary. Earlier versions of the compiler
> ignored those following
> bytes. "
>   
Nice, thanks a lot Anders.

I will raise a bug report on the documentation now; this should be clear 
both in the asn1ct docs (which it is ~is, the options list doesn't 
mention it but the description does) and in the asn1rt docs (no mention 
of the alternative result). However at least this list discussion will 
serve to document this (and my oversight) for others in the future :-)

As an aside, has anyone noticed/looked at the UTF8 functions in the 
asn1rt module? I've not seen them mentioned in any of the Unicode 
discussions yet.

Regards,
Bruce
 


From christophe.romain@REDACTED  Mon Oct  1 10:45:51 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Mon, 1 Oct 2007 10:45:51 +0200
Subject: [erlang-questions] Erlang release management
In-Reply-To: <f69c840c0709281710n2e7bdedfkaedaeafd3888d7e4@mail.gmail.com>
References: <d791b8790709271221s45b5aa58qe230c85e359f7b85@mail.gmail.com>
	<m1ir5v3iy1.fsf@aegir.wgtn.cat-it.co.nz>
	<4B3109BF-2CE8-4768-A996-AC3188F1AF5C@process-one.net>
	<d791b8790709280730h5487328epf3ca89beb283b316@mail.gmail.com>
	<f69c840c0709281710n2e7bdedfkaedaeafd3888d7e4@mail.gmail.com>
Message-ID: <9D1F5713-B4DE-4866-87A4-1092CA27AFAF@process-one.net>

distributed updates is a planed feature for CEAN 1.4.
that's why i said "should". handling release tarballs should also be  
doable.

by now, cean:upgrade just installs and reloads new version of code on  
the local node.
in next version it will run on a set of nodes.
in that stage, it is certainly not suited enough for production live  
systems with loop and state data (the application is not aware of  
code change, we just tell erlang to purge the old code)

for that reason, concerning live code upgrade i have no good solution  
by now.
  - if applying only on local node we may just call  
AppModule:code_change (if exists)
  - if applying on cluster, it may be possible to shut-down one node,  
upgrade code, restart-node. (in that case one must use a start/stop  
script). then do it again and again, for each node.

Eric, may sinan be able, at build stage, to generate some kind of  
generic code reload mecanism developer can plug its application loop  
on ?



From opendev@REDACTED  Mon Oct  1 19:32:20 2007
From: opendev@REDACTED (Joern)
Date: Mon, 1 Oct 2007 19:32:20 +0200
Subject: [erlang-questions] gen_tcp custom framing
Message-ID: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>

Dear erlang-questions,

having to deal with a non-inet supported packet format on TCP / IP
(STX / ETX limited, fixed size header, 7-bit encoded payload length) I
am wondering what the best practice might be. Right now I spawn_link
into a passive recv loop - is it possible to add a custom framing
behaviour to inet and get to use { active, once }?

Best regards,


rgs/joern
--


From gbulmer@REDACTED  Mon Oct  1 23:19:08 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Mon, 1 Oct 2007 22:19:08 +0100
Subject: [erlang-questions] Regular expression library (was Not an
	Erlang fan)
In-Reply-To: <3dbc6d1c0709271529s33a1d36fya744ec6e398349a9@mail.gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<A0DD3EF2-4DFB-4084-9FD7-C1999FA98654@gmail.com>
	<3dbc6d1c0709271529s33a1d36fya744ec6e398349a9@mail.gmail.com>
Message-ID: <9B315724-7CDD-4C0E-A492-82DA2426DD18@gmail.com>

Robert

Sorry for the tardy response. I've been enjoying Erlang too much ... :-)

The results for your new version on pathological cases look outstanding.

How does this versions performance compare to the existing regexp for  
simpler matches (without back references)?

G Bulmer

PS - See y'all in Freiburg on Thursday & Friday

> On 27/09/2007, G Bulmer wrote:
>
> I guess it's the 'pathological' cases that worry me; "MY software
> NEVER breaks on the easy cases" :-)
> Seriously though, having a regexp with stable behaviour wins over one
> that gets the last 200% of performance *most of the time*, but is
> unstable.
>
> I did some tests using Russ Cox example comparing the old regexp  
> module (not a good comparison i know) and a new version I am  
> working on. The results confirmed his findings.
>
>  N              15        18         20         22           
> 25         30          40
> regexp        40      360      1570      6900     59000
> re              0.2      0.25      0.32      0.38        0.5         
> 0.73        1.4
>
> All times in millisecs. The old regexp uses a backtracking algorithm.
>
> Which tends to show that we will have no problems with pathological  
> cases. :-) If people feel that it is too slow for the simpler  
> regexps and would prefer to use a C library then it is definitely  
> important to choose the *right* library.
>
> Robert
>



From doug.currie@REDACTED  Mon Oct  1 23:37:14 2007
From: doug.currie@REDACTED (Doug Currie)
Date: Mon, 1 Oct 2007 17:37:14 -0400
Subject: [erlang-questions] garbage collection questions
Message-ID: <347896279.20071001173714@gmail.com>

Hi,

Digging into Erlang a bit, I was dismayed by the use of get & put in
the shootout benchmark nsieve
http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsieve&lang=hipe&id=0

I noted the use of ets and then hipe bytearray in
http://www.nabble.com/shootout---nsieve-bits-benchmark-t1487923.html

But I am running on Windows, so no HiPE :-(

Evaluating the use of integers to replace bytearray just for fun (I
was not expecting it to be competitive), I ran into GC problems.

Here is the code:

main(N) when N >= 2 -> gcst(), ns(N), gcst(), gcst(), ns(N-1), gcst(), ns(N-2), gcst().

gcst() -> 
    io:fwrite("GC: ~p~n", [statistics(garbage_collection)]). 

ns(N) ->
    M = (1 bsl N)*10000,
    io:fwrite("Primes up to ~8.10B ~8.10B~n", [M, ns(0, 2, M, 0)]).

ns(_, I, M, C) when I > M -> C;
ns(B, I, M, C) ->
    case is_prime(B, I) of
        true  -> ns(mark(B, I, M, I+I), I+1, M, C+1);
        false -> ns(B, I+1, M, C)
    end.

mark(B, _, M, K) when K > M -> B;
mark(B, I, M, K) -> mark(not_prime(B, K), I, M, K+I).

is_prime(B, I) ->  0 =:= ((1 bsl I) band B).

not_prime(B, K) -> ((1 bsl K) bor B).

%%%

Here is what happens:

Erlang (BEAM) emulator version 5.5.5 [async-threads:0]

Eshell V5.5.5  (abort with ^G)
1> cd("../../../..").
C:/Dev/cean
ok
2> c(nsieve_e).
{ok,nsieve_e}
3> timer:tc(nsieve_e,main,[6]).
GC: {412,513637,0}
Primes up to   640000    52074
GC: {4540,144952244,0}
GC: {4541,144952601,0}
Primes up to   320000    27608
GC: {6659,4192756888,0}
Primes up to   160000    14683
GC: {7400,4055981814,0}
{858925000,ok}
4> timer:tc(nsieve_e,main,[7]).
GC: {7410,4055985246,0}

Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot allocate 373662860 bytes of memory (of type "heap").


Abnormal termination

%%

The code produces the correct answers (slowly), and should not
accumulate garbage since the loops are all tail recursive.

Two questions:

Why isn't the garbage being collected inside my ns (or mark) loop?

How does the Words_Reclaimed go down between:

GC: {6659,4192756888,0}
Primes up to   160000    14683
GC: {7400,4055981814,0}

?

Thanks.

e

-- 
Doug Currie
Londonderry, NH, USA



From james.hague@REDACTED  Tue Oct  2 00:28:06 2007
From: james.hague@REDACTED (James Hague)
Date: Mon, 1 Oct 2007 17:28:06 -0500
Subject: [erlang-questions] garbage collection questions
In-Reply-To: <347896279.20071001173714@gmail.com>
References: <347896279.20071001173714@gmail.com>
Message-ID: <e7cc45650710011528j66f1a7f2t50d31c41baefe941@mail.gmail.com>

>Digging into Erlang a bit, I was dismayed by the use of get & put in
> the shootout benchmark nsieve

Originally the sieve benchmark used lists--which results in some
pretty Erlang code--but there wasn't consistency between languages.
Some functional languages used lists for the sieve benchmark, most
other entries used destructive array updates.  I believe the point of
nsieve was to require destructive array updates.


From jao@REDACTED  Tue Oct  2 04:09:02 2007
From: jao@REDACTED (Jack Orenstein)
Date: Mon, 1 Oct 2007 22:09:02 -0400
Subject: [erlang-questions] Trying to use gen_leader and gdict
Message-ID: <E0A0DD86-5087-4349-B3A6-40219627A949@geophile.com>

I am running Erlang R11B on OS X and have been investigating
gen_leader.  I'm having some trouble with the gdict example included
with gen_leader. I wrote the following program to use gdict:

     -module(test).
     -export([main/0]).

     -define(DUMP(X), io:format("~p:~p - ~p = ~p~n", [?MODULE, ? 
LINE, ??X, X])).

     main() ->
         Nodes = [node()],
         {ok, D} = gdict:new(test, Nodes, Nodes),
         ?DUMP(D),
         ?DUMP(gdict:append(a, 1, D)),
         ?DUMP(gdict:append(b, 2, D)),
         ?DUMP(gdict:append(c, 3, D)),
         ?DUMP(gdict:find(a, D)),
         ?DUMP(gdict:find(b, D)),
         ?DUMP(gdict:find(c, D)).

This causes a timeout:

     zack$ time erl -sname z -noshell -s test main -s init stop
     init({dict,0,
                16,
                16,
                8,
                80,
                48,
                {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}})
     {"init terminating in do_boot",{timeout,{gen_leader,leader_call, 
[<0.34.0>,{store,#Fun<gdict.0.10659195>}]}}}

     Crash dump was written to: erl_crash.dump
     init terminating in do_boot ()

     real	0m5.564s
     user	0m0.185s
     sys	0m0.360s

Can anyone see what I'm doing wrong?

Jack Orenstein



From bjorn@REDACTED  Tue Oct  2 07:29:32 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 02 Oct 2007 07:29:32 +0200
Subject: [erlang-questions] garbage collection questions
In-Reply-To: <347896279.20071001173714@gmail.com>
References: <347896279.20071001173714@gmail.com>
Message-ID: <o1ve9qt77n.fsf@erix.ericsson.se>

Doug Currie <doug.currie@REDACTED> writes:

> 
> Two questions:
> 
> Why isn't the garbage being collected inside my ns (or mark) loop?

For historical reasons, arithmetic operations never causes a
garbage collection. Instead, if a an arithmetic operation produces
a float or bignum, a "heap fragment" outside of the heap is allocated
for it. Too much data in heap fragments will eventually trigger a GC,
though, but in this case too late.

In R12B, we will change that. Bignums and floats will be put onto the
heap and there will be a garbage collection if the heap is full.

Testing your program in our development version of R12B (I only had
patience to use the argument 3), I see that there are a lot more GCs
but that it actually runs faster than in R11B-5.


By the way, your program will run faster if you define is_prime/2
like this:

is_prime(B, I) ->  0 =:= ((B bsr I) band 1).

The reason is that at most one bignum will be constructed, not two.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From ulf.wiger@REDACTED  Tue Oct  2 09:40:21 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Tue, 02 Oct 2007 09:40:21 +0200
Subject: [erlang-questions] Trying to use gen_leader and gdict
In-Reply-To: <E0A0DD86-5087-4349-B3A6-40219627A949@geophile.com>
References: <E0A0DD86-5087-4349-B3A6-40219627A949@geophile.com>
Message-ID: <4701F5E5.2080109@ericsson.com>

Jack Orenstein wrote:
> I am running Erlang R11B on OS X and have been investigating
> gen_leader.  I'm having some trouble with the gdict example included
> with gen_leader. I wrote the following program to use gdict:
> ...
> 
> This causes a timeout:
> 
>      zack$ time erl -sname z -noshell -s test main -s init stop
>      init({dict,0,
>                 16,
>                 16,
>                 8,
>                 80,
>                 48,
>                 {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
>                 {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}})
>      {"init terminating in do_boot",{timeout,{gen_leader,leader_call, 
> [<0.34.0>,{store,#Fun<gdict.0.10659195>}]}}}
> ...
> 
> Can anyone see what I'm doing wrong?

You're not doing anything wrong. There's a bug in gen_leader.
When you start gen_leader with only one candidate node, it
will still go into a receive loop waiting for responses from
other candidates.

I have a patched version of gen_leader:

svn co http://svn.ulf.wiger.net/gproc/patches/gen_leader

It has a few other unverified* patches, such as the ability
to start gen_leader with no nodes at all (local-only mode) and
then triggering leader election by the callback returning
{activate, Cands, ...}.

* Note that gen_leader is very well tested (except, obviously,
for the case where you start gen_leader with only one candidate).
So "unverified" here means that I've only put this code through
very limited testing.

BR,
Ulf W





From hio@REDACTED  Tue Oct  2 12:17:20 2007
From: hio@REDACTED (YAMASHINA Hio)
Date: Tue, 2 Oct 2007 19:17:20 +0900
Subject: [erlang-questions] why there is no function proc_lib:start(Fun)?
Message-ID: <20071002191720.0207f379.hio@hio.jp>


Hi all.

When using proc_lib:start, 
I force to export the entry point function
even if there is no need to invoke from
outside of that module.

There is proc_lib:spawn(Fun) function.
Why proc_lib:start(Fun) does not exist?


Regards.

-- 
YAMASHINA Hio <hio@REDACTED>



From vito.detullio@REDACTED  Tue Oct  2 14:16:10 2007
From: vito.detullio@REDACTED (ZeD)
Date: Tue, 02 Oct 2007 14:16:10 +0200
Subject: [erlang-questions] [LONG] How to make synchronous message passing
Message-ID: <fdtc5e$ec0$1@sea.gmane.org>

Hi all.

For my studies, I'm trying to implement a way to let me do synchronous
message passing, but I found some problems.

ATM I'm trying starting from a simple idea: make an asynchronous
communication, but wait for a reply

synchro(Pid, Message) ->
    Pid ! {self(), Message},
    receive
        Reply ->
            Reply
    end.

For simple communications it works flawlessly, but I encountered problems
when I tried to make talk together a net of pairs (processes which are
istantiations of the same function f)

f(Name) ->
  receive
    {Sender, {do, Pid, Do}} ->
        Sender ! synchro(Pid, Do),
        f(Name);
    {Sender, {ask, Pid}} ->
       Sender ! synchro(Pid, name),
       f(Name);
    {Sender, name} ->
       Sender ! Name,
       f(Name)
  end.

for istance, let's say a process P want to make Q talk to P himself (for
some reason). So, we shoud write

synchro(P, {do, Q, {ask, P}}).

but, instead of the Name of P, all I have is "{P, name}", because the
synchro function taked the "wrong" reply (as it wasn't a reply to a
communication, just another message.

So I tried to restrict the reply on a specific form:

synchro2(Pid, Message) ->
    Pid ! {self(), Message},
    receive
        {Pid, Message, Reply} ->
            Reply
    end.

f2(Name) ->
  receive
    {Sender, {do, Pid, Do}=M} ->
        Sender ! {self(), M, synchro(Pid, Do)},
        f2(Name);
    {Sender, {ask, Pid}=M} ->
       Sender ! {self(), M, synchro(Pid, name)},
       f2(Name);
    {Sender, name} ->
       Sender ! {self(), name, Name},
       f2(Name)
  end.

but all I got is a blocked session, because while P wait for a reply from Q,
he got instead another query, and lost it, so Q wait for P and P for Q.

So, how can I implement an "atomic" (?) synchronous message passing? I tried
to "collect" all the non-matching messages in synchro, and resend them to
self(), but nothing changed.

-- 
Under construction



From vladdu55@REDACTED  Tue Oct  2 14:20:26 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 2 Oct 2007 14:20:26 +0200
Subject: [erlang-questions] [LONG] How to make synchronous message passing
In-Reply-To: <95be1d3b0710020520r369afb43h57115ddb6b810d5d@mail.gmail.com>
References: <fdtc5e$ec0$1@sea.gmane.org>
	<95be1d3b0710020520r369afb43h57115ddb6b810d5d@mail.gmail.com>
Message-ID: <95be1d3b0710020520w76125da3pe6a33de4bb26f939@mail.gmail.com>

Hi,

One way could be to create a unique reference for each call and send
it together with the request. When waiting for the reply, selectively
receive with the reference as parameter.

Something in the lines of:

 synchro(Pid, Message) ->
     Ref = make_ref(),
     Pid ! {self(), Ref, Message},
     receive
         {Ref, Reply} ->
             Reply
     end.

Instead of Pid ! Reply, call

  reply(Pid, Ref, Reply) ->
     Pid ! {Ref, Reply).

best regards,
Vlad

On 10/2/07, ZeD <vito.detullio@REDACTED> wrote:
> Hi all.
>
> For my studies, I'm trying to implement a way to let me do synchronous
> message passing, but I found some problems.
>
> ATM I'm trying starting from a simple idea: make an asynchronous
> communication, but wait for a reply
>
> synchro(Pid, Message) ->
>     Pid ! {self(), Message},
>     receive
>         Reply ->
>             Reply
>     end.
>
> For simple communications it works flawlessly, but I encountered problems
> when I tried to make talk together a net of pairs (processes which are
> istantiations of the same function f)
>
> f(Name) ->
>   receive
>     {Sender, {do, Pid, Do}} ->
>         Sender ! synchro(Pid, Do),
>         f(Name);
>     {Sender, {ask, Pid}} ->
>        Sender ! synchro(Pid, name),
>        f(Name);
>     {Sender, name} ->
>        Sender ! Name,
>        f(Name)
>   end.
>
> for istance, let's say a process P want to make Q talk to P himself (for
> some reason). So, we shoud write
>
> synchro(P, {do, Q, {ask, P}}).
>
> but, instead of the Name of P, all I have is "{P, name}", because the
> synchro function taked the "wrong" reply (as it wasn't a reply to a
> communication, just another message.
>
> So I tried to restrict the reply on a specific form:
>
> synchro2(Pid, Message) ->
>     Pid ! {self(), Message},
>     receive
>         {Pid, Message, Reply} ->
>             Reply
>     end.
>
> f2(Name) ->
>   receive
>     {Sender, {do, Pid, Do}=M} ->
>         Sender ! {self(), M, synchro(Pid, Do)},
>         f2(Name);
>     {Sender, {ask, Pid}=M} ->
>        Sender ! {self(), M, synchro(Pid, name)},
>        f2(Name);
>     {Sender, name} ->
>        Sender ! {self(), name, Name},
>        f2(Name)
>   end.
>
> but all I got is a blocked session, because while P wait for a reply from Q,
> he got instead another query, and lost it, so Q wait for P and P for Q.
>
> So, how can I implement an "atomic" (?) synchronous message passing? I tried
> to "collect" all the non-matching messages in synchro, and resend them to
> self(), but nothing changed.
>
> --
> Under construction
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From casper2000a@REDACTED  Tue Oct  2 14:43:29 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Tue, 2 Oct 2007 18:13:29 +0530
Subject: [erlang-questions] Distrobuted/Fragmented Mnesia node crash
In-Reply-To: <fdtc5e$ec0$1@sea.gmane.org>
References: <fdtc5e$ec0$1@sea.gmane.org>
Message-ID: <009401c804f1$d59b66e0$80d234a0$@com>

Hi,

I run 3 distributed Erlang nodes, dedicated to run Mnesia database server.
Another Erlang node runs the application which read/write to those tables.
Schema is created as disk_copies in all 4 nodes and tables are created as
fragmented and distributed tables between 3 dedicated Mnesia nodes. I.e. 99
fragments of a table distributed in 3 nodes resulting 33 fragments in each.
There're about 250-350 Db write/s and 500-800 DB read/s

Performance is quite good. I don't see "Mnesia overloaded" warnings. However
occasionally one of the Mnesia node crashes. Sometimes the time between 2
crashes of the same Mnesia node may be about 1-2 days, while the next crash
takes about 10-20 days to happen. It doesn't even generate the crash_dump
file or even if its generated, the size is 0 bytes.

Any idea what could be the cause? Are others also experience the same? I
don't see any abnormal Load, Processor or Memory usage when these crashes
occurs.

BRgds,
- Eranga




From richardc@REDACTED  Tue Oct  2 14:54:49 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Tue, 02 Oct 2007 14:54:49 +0200
Subject: [erlang-questions] [LONG] How to make synchronous message
	passing
In-Reply-To: <fdtc5e$ec0$1@sea.gmane.org>
References: <fdtc5e$ec0$1@sea.gmane.org>
Message-ID: <47023F99.6040201@it.uu.se>

ZeD wrote:
> for istance, let's say a process P want to make Q talk to P himself (for
> some reason). So, we shoud write
> 
> synchro(P, {do, Q, {ask, P}}).

It seems that you are not talking about normal synchronous communication,
but rather a form of nested request/reply. In synchronous message passing,
if P is waiting for a reply from Q then he is by definition blocked and
cannot answer the query from Q. Your implementation in synchro2 is correct,
but synchronous communication is not what you need for this problem. You
have to rethink what f does.

     /Richard



From bbmaj7@REDACTED  Tue Oct  2 15:22:07 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Tue, 2 Oct 2007 23:22:07 +1000 (EST)
Subject: [erlang-questions] nodes not visible until net_adm:world()
Message-ID: <678455.59630.qm@web52002.mail.re2.yahoo.com>

I find that the following calls do not work as I expect.
When I start a new node and already have several nodes running...

nodes() returns []

global:registered_names() returns []

Cannot send to global registered processes.

... But then after I do net_adm:world() or net_adm:world_list([myhost]),
suddenly all the nodes and registered names are visible. It's as if the VM is
not trying to contact epmd until after world() or world_list() is called.

This is observed on the first call to either of these

=PROGRESS REPORT==== 2-Oct-2007::23:19:21 ===
          supervisor: {local,inet_gethost_native_sup}
             started: [{pid,<0.374.0>},{mfa,{inet_gethost_native,init,[[]]}}]

=PROGRESS REPORT==== 2-Oct-2007::23:19:21 ===
          supervisor: {local,kernel_safe_sup}
             started: [{pid,<0.373.0>},
                       {name,inet_gethost_native_sup},
                       {mfa,{inet_gethost_native,start_link,[]}},
                       {restart_type,temporary},
                       {shutdown,1000},
                       {child_type,worker}]

Is this behaviour expected; and if so can someone explain what the proper
mechanism is to start the node so epmd facilities are available?

I'm running R11B5 under Linux.

--
  Rich



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From klacke@REDACTED  Tue Oct  2 20:22:57 2007
From: klacke@REDACTED (=?ISO-8859-1?Q?Claes_Wikstr=F6m?=)
Date: Tue, 02 Oct 2007 20:22:57 +0200
Subject: [erlang-questions] nodes not visible until net_adm:world()
In-Reply-To: <678455.59630.qm@web52002.mail.re2.yahoo.com>
References: <678455.59630.qm@web52002.mail.re2.yahoo.com>
Message-ID: <47028C81.1020604@hyber.org>

Richard Andrews wrote:

> 
> Is this behaviour expected; and if so can someone explain what the proper
> mechanism is to start the node so epmd facilities are available?
> 

Node to node connections are on-demand, thus not until you
actively try to spawn/send to a remote node is connection
established. Hence originally nodes() == [], and then if you do

{foo, name@REDACTED} ! whatever

will a connection to name@REDACTED be established and if all
is well, nodes() == [name@REDACTED]

/klacke



From H.Li@REDACTED  Tue Oct  2 22:34:53 2007
From: H.Li@REDACTED (H.Li@REDACTED)
Date: Tue, 2 Oct 2007 21:34:53 +0100 (BST)
Subject: [erlang-questions] An update of Wranlger
Message-ID: <3089.88.110.130.21.1191357293.squirrel@webmail.cs.kent.ac.uk>

Dear Erlangers,

We are pleased to announce that an updated version of Wrangler with a new
refactoring (move a function definition between modules) and a few bug
fixes is now available from our project webpage:

     http://www.cs.kent.ac.uk/projects/forse/.

Along with the update of Wrangler, we have also updated our wiki where we
document our work and thoughts on refactoring. You can find it at:

 https://www.cs.kent.ac.uk/projects/refactor-fp/fp-design/RefactorErlang

We are glad to let you know that after six-month suspension, we have now
restarted the implementation of Wrangler. While there is a long list of
refactorings waiting us to implement, such as duplicated code elimination,
add an argument to a function definition, introduce/remove processes,
etc, your thoughts and suggestions are very much welcomed. Our project
email is: erlang-refactor@REDACTED

We have been working with the Erlang researchers at the E?tv?s Lor?nd
Unversity in Hungary, who have also been looking into refactorings and
announced their first release recently, and we hope in due course to make
a joint release of our work.

Best Regards,
Simon Thompson, Huiqing Li



From dougedmunds@REDACTED  Wed Oct  3 00:47:21 2007
From: dougedmunds@REDACTED (Doug Edmunds)
Date: Tue, 2 Oct 2007 15:47:21 -0700
Subject: [erlang-questions] TouchGraph Google Browser
Message-ID: <f6e9854a0710021547o6bfd26c8v12d4205a1b102a2e@mail.gmail.com>

I just ran across this on the web:

http://www.touchgraph.com/TGGoogleBrowser.html

type "erlang" in the box  then click the 'Graph it' button.

-- dae
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071002/b3019b0a/attachment.htm>

From bbmaj7@REDACTED  Wed Oct  3 01:39:24 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Wed, 3 Oct 2007 09:39:24 +1000 (EST)
Subject: [erlang-questions] node discovery (was nodes not visible until
	net_adm:world() )
In-Reply-To: <47028C81.1020604@hyber.org>
Message-ID: <275218.36336.qm@web52009.mail.re2.yahoo.com>


--- Claes Wikstr?m <klacke@REDACTED> wrote:

> Node to node connections are on-demand, thus not until you
> actively try to spawn/send to a remote node is connection
> established. Hence originally nodes() == [], and then if you do
> 
> {foo, name@REDACTED} ! whatever
> 
> will a connection to name@REDACTED be established and if all
> is well, nodes() == [name@REDACTED]

It seems the new node needs to be connected to a least one node in the global
nameserver network before any of the global registration information can be
used. Once connected to the first node, the rest become visible automatically.
However the new node should not know any other node names a-priori.

Is there a more appropriate mechanism to achieve a fully connected network than
net_adm:world()? I find that call distasteful as the connected network is a
side-effect of the call rather than the intent and as such might not always
work that way in the future. Is there any call which has the explicit intent of
creating a fully connected network?



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From jcone@REDACTED  Wed Oct  3 07:21:49 2007
From: jcone@REDACTED (James Cone)
Date: Wed, 03 Oct 2007 18:21:49 +1300
Subject: [erlang-questions] Bad value on output port 'tcp_inet'
Message-ID: <470326ED.6020107@eservglobal.com>

I'm having this, which was discussed last year, in R11B-5 on Windows XP.

I've eliminated one theory: I can still have it even when each process 
owns at most one TCP socket, and only that process talks to that socket.

Advice welcome about how to make high throughputs on tcp sockets safe.


From vito.detullio@REDACTED  Wed Oct  3 08:53:21 2007
From: vito.detullio@REDACTED (ZeD)
Date: Wed, 03 Oct 2007 08:53:21 +0200
Subject: [erlang-questions] [LONG] How to make synchronous message
	passing
References: <fdtc5e$ec0$1@sea.gmane.org> <47023F99.6040201@it.uu.se>
Message-ID: <fdvdjv$jac$1@sea.gmane.org>

Richard Carlsson wrote:

>> synchro(P, {do, Q, {ask, P}}).
> It seems that you are not talking about normal synchronous communication,
> but rather a form of nested request/reply. In synchronous message passing,
> if P is waiting for a reply from Q then he is by definition blocked and
> cannot answer the query from Q. Your implementation in synchro2 is
> correct, but synchronous communication is not what you need for this
> problem. You have to rethink what f does.

mmm

maybe I made a wrong example, or maybe you are right (effectively using
asinchronous communications I can have a reply)... let me change the
question:

if a process P want to make a synchronous call to Q, then it send a message 
and waits for a reply.

What's appened when another process, R (just for make and example) send a
message to P 'before' the Q reply?
In particular, what if Q first send another message to P, then waits for the
reply, and only after all this, send the reply to the original message to
P?

Oh, and just to know, let's say P was described by the function f(), which
have a receive; now P want to talk to Q, then now the behavior of P is
described by synchro2(), wich also have a receive. Let's say P finally had
a reply, so the control came back to f(). What append to messages not
matched by synchro2()? they became automatically "receveived" by f()? They
get lost?

-- 
Under construction



From richardc@REDACTED  Wed Oct  3 09:27:13 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Wed, 03 Oct 2007 09:27:13 +0200
Subject: [erlang-questions] [LONG] How to make synchronous
	message	passing
In-Reply-To: <fdvdjv$jac$1@sea.gmane.org>
References: <fdtc5e$ec0$1@sea.gmane.org> <47023F99.6040201@it.uu.se>
	<fdvdjv$jac$1@sea.gmane.org>
Message-ID: <47034451.80900@it.uu.se>

ZeD wrote:
> if a process P want to make a synchronous call to Q, then it send a message 
> and waits for a reply.
> 
> What's appened when another process, R (just for make and example) send a
> message to P 'before' the Q reply?

Then, the message from R will be before the reply from Q in the mailbox
of P. But if you use selective matching, as in synchro2, then P can
ignore the message from R and accept only the message from Q.

> In particular, what if Q first send another message to P, then waits for the
> reply, and only after all this, send the reply to the original message to
> P?

If P is in synchro2, and Q sends another message to P (not the reply),
then either P will mistakenly think that it is the reply (if it can have
the same form as a reply), or P will just ignore the message (like it
ignored the message from R) and keep waiting for the reply. This is
the whole point: if the communication is _synchronous_, then P will not
do anything at all (per definition) until it gets the reply. Other
messages will be queued up in the mailbox.

> Oh, and just to know, let's say P was described by the function f(), which
> have a receive; now P want to talk to Q, then now the behavior of P is
> described by synchro2(), wich also have a receive. Let's say P finally had
> a reply, so the control came back to f(). What append to messages not
> matched by synchro2()? they became automatically "receveived" by f()? They
> get lost?

No. The process P has a mailbox (a queue), and when the receive-
expression in synchro2 finishes, it will only remove the matched
message. All other messages remain in the queue. Nothing is lost
unless you throw it away yourself.

    /Richard


From adam@REDACTED  Wed Oct  3 09:31:53 2007
From: adam@REDACTED (Adam Lindberg)
Date: Wed, 3 Oct 2007 09:31:53 +0200
Subject: [erlang-questions] [LONG] How to make synchronous message
	passing
In-Reply-To: <fdvdjv$jac$1@sea.gmane.org>
References: <fdtc5e$ec0$1@sea.gmane.org> <47023F99.6040201@it.uu.se>
	<fdvdjv$jac$1@sea.gmane.org>
Message-ID: <6344005f0710030031l123bca1fj72407f45eda44561@mail.gmail.com>

What you are doing by using a synchronous sending mechanism as described
above AND letting both P and Q send requests to each other is effectively
implementing dead lock. Because messages will cross, sooner or later, and if
both P and Q can send requests to each other and these requests cross in
between you have a dead lock. You should consider what you really want to
achieve with synchronous message passing (or rather synchronous requests,
there are no such thing as synchronous message passing, it is one way only).

Synchronous requests can be desirable in certain situations, but almost
never in a P <-> Q relation. Then you would also need atomic synchronous
message requests. That could maybe be achieved but very unlikely what you
really want.

Cheers!
Adam

On 10/3/07, ZeD <vito.detullio@REDACTED> wrote:
>
> Richard Carlsson wrote:
>
> >> synchro(P, {do, Q, {ask, P}}).
> > It seems that you are not talking about normal synchronous
> communication,
> > but rather a form of nested request/reply. In synchronous message
> passing,
> > if P is waiting for a reply from Q then he is by definition blocked and
> > cannot answer the query from Q. Your implementation in synchro2 is
> > correct, but synchronous communication is not what you need for this
> > problem. You have to rethink what f does.
>
> mmm
>
> maybe I made a wrong example, or maybe you are right (effectively using
> asinchronous communications I can have a reply)... let me change the
> question:
>
> if a process P want to make a synchronous call to Q, then it send a
> message
> and waits for a reply.
>
> What's appened when another process, R (just for make and example) send a
> message to P 'before' the Q reply?
> In particular, what if Q first send another message to P, then waits for
> the
> reply, and only after all this, send the reply to the original message to
> P?
>
> Oh, and just to know, let's say P was described by the function f(), which
> have a receive; now P want to talk to Q, then now the behavior of P is
> described by synchro2(), wich also have a receive. Let's say P finally had
> a reply, so the control came back to f(). What append to messages not
> matched by synchro2()? they became automatically "receveived" by f()? They
> get lost?
>
> --
> Under construction
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071003/f09ae79f/attachment.htm>

From bengt.kleberg@REDACTED  Wed Oct  3 09:50:47 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 03 Oct 2007 09:50:47 +0200
Subject: [erlang-questions] =?iso-8859-1?q?Surprising_conclusion_in_=B7_Pa?=
 =?iso-8859-1?q?rallel_programming_environments=3A_less_is_more?=
Message-ID: <470349D7.2050108@ericsson.com>

greetings,

http://blogs.intel.com/research/2007/10/parallel_programming_environme.html
has a rather surprising conclusion to solving parallel programming 
problems. no new languages are needed, instead someone should fix the 
existing ones.


bengt
-- 
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


From chandrashekhar.mullaparthi@REDACTED  Wed Oct  3 10:12:26 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 3 Oct 2007 09:12:26 +0100
Subject: [erlang-questions] Bad value on output port 'tcp_inet'
In-Reply-To: <470326ED.6020107@eservglobal.com>
References: <470326ED.6020107@eservglobal.com>
Message-ID: <b447a30d0710030112p75743a68l22a4c905780d9eb6@mail.gmail.com>

On 03/10/2007, James Cone <jcone@REDACTED> wrote:
> I'm having this, which was discussed last year, in R11B-5 on Windows XP.
>
> I've eliminated one theory: I can still have it even when each process
> owns at most one TCP socket, and only that process talks to that socket.
>
> Advice welcome about how to make high throughputs on tcp sockets safe.

This is most probably because you are sending something other than a
(deep) list of integers or binaries. Check the data you are trying to
send down the socket.

Chandru


From klacke@REDACTED  Wed Oct  3 10:20:37 2007
From: klacke@REDACTED (Claes Wikstrom)
Date: Wed, 03 Oct 2007 10:20:37 +0200
Subject: [erlang-questions] Bad value on output port 'tcp_inet'
In-Reply-To: <b447a30d0710030112p75743a68l22a4c905780d9eb6@mail.gmail.com>
References: <470326ED.6020107@eservglobal.com>
	<b447a30d0710030112p75743a68l22a4c905780d9eb6@mail.gmail.com>
Message-ID: <470350D5.4090800@hyber.org>

Chandru wrote:

> 
> This is most probably because you are sending something other than a
> (deep) list of integers or binaries. Check the data you are trying to
> send down the socket.
> 

Typically atoms,

is in for example:

gen_tcp:send(Socket, [Bin1, "foo", some_atom, Bin2]).

/klacke


From chandrashekhar.mullaparthi@REDACTED  Wed Oct  3 10:44:31 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 3 Oct 2007 09:44:31 +0100
Subject: [erlang-questions] node discovery (was nodes not visible until
	net_adm:world() )
In-Reply-To: <275218.36336.qm@web52009.mail.re2.yahoo.com>
References: <47028C81.1020604@hyber.org>
	<275218.36336.qm@web52009.mail.re2.yahoo.com>
Message-ID: <b447a30d0710030144y3e0e0d67m11bfcb445be17d0f@mail.gmail.com>

On 03/10/2007, Richard Andrews <bbmaj7@REDACTED> wrote:
>
> Is there a more appropriate mechanism to achieve a fully connected network than
> net_adm:world()? I find that call distasteful as the connected network is a
> side-effect of the call rather than the intent and as such might not always
> work that way in the future. Is there any call which has the explicit intent of
> creating a fully connected network?
>

The kernel has a couple of configuration parameters which might
achieve what you want.

Lookup 'sync_nodes_optional'  and 'sync_nodes_mandatory' in the kernel man page.

cheers
Chandru


From chadrwilson@REDACTED  Wed Oct  3 16:46:29 2007
From: chadrwilson@REDACTED (Chad Wilson)
Date: Wed, 3 Oct 2007 10:46:29 -0400
Subject: [erlang-questions]
	=?iso-8859-1?q?Surprising_conclusion_in_=B7_Pa?=
	=?iso-8859-1?q?rallel_programming_environments=3A_less_is_more?=
In-Reply-To: <470349D7.2050108@ericsson.com>
References: <470349D7.2050108@ericsson.com>
Message-ID: <c8d225930710030746s263474d8r8936f8a01791ef06@mail.gmail.com>

Here is a link to the full article.

http://www.columbia.edu/~ss957/whenchoice.html

-w


From joelr1@REDACTED  Wed Oct  3 19:48:53 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Wed, 3 Oct 2007 18:48:53 +0100
Subject: [erlang-questions] Using Lisp to extend a Mac IDE for Erlang
Message-ID: <AB032F81-4F53-4ECD-9E8C-B63AE79C04A6@gmail.com>

I'm between projects right now (anyone want to hire me?) so I have  
time to spare. I'm once again thinking of the Mac IDE for Erlang.  
More specifically, I'm thinking of the language used to extend such  
an IDE.

I'm wondering if there are any objections to an environment that  
reuses the Distel code and can be extended in Lisp. Sort of like a  
TextMate for Erlang with Lisp instead of Ruby.

I already own a LispWorks license and it comes with a Cocoa bridge so  
the only thing I would need to do to bootstrap is port Distel and  
hook it up to an editor. What do you think?

	Thanks, Joel

--
http://wagerlabs.com







From tomas.abrahamsson@REDACTED  Wed Oct  3 19:24:59 2007
From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson)
Date: Wed, 3 Oct 2007 19:24:59 +0200
Subject: [erlang-questions] why there is no function proc_lib:start(Fun)?
In-Reply-To: <20071002191720.0207f379.hio@hio.jp>
References: <20071002191720.0207f379.hio@hio.jp>
Message-ID: <b330698a0710031024h6e0e75b4ode5c77944e5d8a0f@mail.gmail.com>

On 10/2/07, YAMASHINA Hio <hio@REDACTED> wrote:
> When using proc_lib:start,
> I force to export the entry point function
> even if there is no need to invoke from
> outside of that module.
>
> There is proc_lib:spawn(Fun) function.
> Why proc_lib:start(Fun) does not exist?

I often wanted a proc_lib:start(Fun), too, but
one can use erlang:apply/2 as a workaround:

    proc_lib:start(erlang, apply, [fun() -> proc_lib:init_ack({ok, self()}),
					    do_something()
				   end,
				   []]).

This can be useful for functions in the
rpc module as well.


/Tomas


From rvirding@REDACTED  Wed Oct  3 23:30:09 2007
From: rvirding@REDACTED (Robert Virding)
Date: Wed, 3 Oct 2007 23:30:09 +0200
Subject: [erlang-questions] Regular expression library (was Not an
	Erlang fan)
In-Reply-To: <9B315724-7CDD-4C0E-A492-82DA2426DD18@gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<A0DD3EF2-4DFB-4084-9FD7-C1999FA98654@gmail.com>
	<3dbc6d1c0709271529s33a1d36fya744ec6e398349a9@mail.gmail.com>
	<9B315724-7CDD-4C0E-A492-82DA2426DD18@gmail.com>
Message-ID: <3dbc6d1c0710031430ycf0571dy6775fa15a45e15a9@mail.gmail.com>

Unfortunately at the moment is a little slower on simple regular expressions
than the old regexp module. It seems like it is more difficult to optimise
for the simple cases when you concentrate on the tough ones.

But I'm working on it. Do the versions which use PCRE or other libraries
work on Windows without a C compiler? At the moment that is all I have with
no other development env than Erlang. It would be interesting to compare.

Robert

On 01/10/2007, G Bulmer <gbulmer@REDACTED> wrote:
>
> Robert
>
> Sorry for the tardy response. I've been enjoying Erlang too much ... :-)
>
> The results for your new version on pathological cases look outstanding.
>
> How does this versions performance compare to the existing regexp for
> simpler matches (without back references)?
>
> G Bulmer
>
> PS - See y'all in Freiburg on Thursday & Friday
>
> > On 27/09/2007, G Bulmer wrote:
> >
> > I guess it's the 'pathological' cases that worry me; "MY software
> > NEVER breaks on the easy cases" :-)
> > Seriously though, having a regexp with stable behaviour wins over one
> > that gets the last 200% of performance *most of the time*, but is
> > unstable.
> >
> > I did some tests using Russ Cox example comparing the old regexp
> > module (not a good comparison i know) and a new version I am
> > working on. The results confirmed his findings.
> >
> >  N              15        18         20         22
> > 25         30          40
> > regexp        40      360      1570      6900     59000
> > re              0.2      0.25      0.32      0.38        0.5
> > 0.73        1.4
> >
> > All times in millisecs. The old regexp uses a backtracking algorithm.
> >
> > Which tends to show that we will have no problems with pathological
> > cases. :-) If people feel that it is too slow for the simpler
> > regexps and would prefer to use a C library then it is definitely
> > important to choose the *right* library.
> >
> > Robert
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071003/80624945/attachment.htm>

From gbulmer@REDACTED  Wed Oct  3 23:51:48 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Wed, 3 Oct 2007 23:51:48 +0200
Subject: [erlang-questions] Regular expression library (was Not an
	Erlang fan)
In-Reply-To: <3dbc6d1c0710031430ycf0571dy6775fa15a45e15a9@mail.gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<A0DD3EF2-4DFB-4084-9FD7-C1999FA98654@gmail.com>
	<3dbc6d1c0709271529s33a1d36fya744ec6e398349a9@mail.gmail.com>
	<9B315724-7CDD-4C0E-A492-82DA2426DD18@gmail.com>
	<3dbc6d1c0710031430ycf0571dy6775fa15a45e15a9@mail.gmail.com>
Message-ID: <7B7AF534-23AD-47F7-8368-D0AF7D1CE392@gmail.com>

> Unfortunately at the moment is a little slower on simple regular  
> expressions than the old regexp module. It seems like it is more  
> difficult to optimise for the simple cases when you concentrate on  
> the tough ones.
>
> But I'm working on it. Do the versions which use PCRE or other  
> libraries work on Windows without a C compiler?
I do not know, and I don't have a Windows machine with me.
I'm in sunny Freiburg with my trusty MacBook, across the road from  
the conference in the 'Design Hotel' Stadtgarten.

> At the moment that is all I have with no other development env than  
> Erlang. It would be interesting to compare.
>
> Robert
>
> On 01/10/2007, G Bulmer <gbulmer@REDACTED> wrote: Robert
>
> Sorry for the tardy response. I've been enjoying Erlang too  
> much ... :-)
>
> The results for your new version on pathological cases look  
> outstanding.
>
> How does this versions performance compare to the existing regexp for
> simpler matches (without back references)?
>
> G Bulmer
>
> PS - See y'all in Freiburg on Thursday & Friday
>
> > On 27/09/2007, G Bulmer wrote:
> >
> > I guess it's the 'pathological' cases that worry me; "MY software
> > NEVER breaks on the easy cases" :-)
> > Seriously though, having a regexp with stable behaviour wins over  
> one
> > that gets the last 200% of performance *most of the time*, but is
> > unstable.
> >
> > I did some tests using Russ Cox example comparing the old regexp
> > module (not a good comparison i know) and a new version I am
> > working on. The results confirmed his findings.
> >
> >  N              15        18         20         22
> > 25         30          40
> > regexp        40      360      1570      6900     59000
> > re              0.2      0.25      0.32      0.38        0.5
> > 0.73        1.4
> >
> > All times in millisecs. The old regexp uses a backtracking  
> algorithm.
> >
> > Which tends to show that we will have no problems with pathological
> > cases. :-) If people feel that it is too slow for the simpler
> > regexps and would prefer to use a C library then it is definitely
> > important to choose the *right* library.
> >
> > Robert
> >
>
>



From rvirding@REDACTED  Wed Oct  3 23:32:11 2007
From: rvirding@REDACTED (Robert Virding)
Date: Wed, 3 Oct 2007 23:32:11 +0200
Subject: [erlang-questions] Regular expression library (was Not an
	Erlang fan)
In-Reply-To: <17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
Message-ID: <3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>

That's not too difficult to do at all. The speed you get would depend on
which regexp family you do it for.

Robert

On 27/09/2007, Yariv Sadan <yarivsadan@REDACTED> wrote:
>
> I was wondering if it would be possible/beneficial to write a regexp
> compiler that takes a regexp string and compiles it into an Erlang
> module.
>
> You would use it as follows:
>
> regexp2:compile("f*o+o?", myregexp, [native]),
> true = myregexp:matches("foo").
>
> This could be a very fast regexp engine.
>
> What do you think?
>
> Yariv
>
> On 9/26/07, Robert Virding <rvirding@REDACTED> wrote:
> > The regular expression module which is included in the distribution is a
> > simple rather na?ve one written in Erlang which uses a back-tracking
> > algorithm directly on the AST. It has all the problems Russ Cox
> mentioned in
> > his paper. I wrote it long ago when no one was interested in parsing
> regexps
> > in Erlang. :-)
> >
> > I am rewriting it at the moment, still in Erlang but using the same
> > algorithm, or similar to the one, which Russ describes. It is about the
> same
> > speed as the existing one for simpler regexps but it can handle all the
> > pathological cases as it should. It will handle POSIX syntax and
> semantics
> > (but no back-references*). There will also be a version which will
> implement
> > a subset of PERL regexps with PERL semantics. Both will work on either
> lists
> > or binaries.
> >
> > I hadn't seen the lambda-the-ultimate discussion you mentioned and will
> > check it out.
> >
> > There are a number of regexp packages for Erlang which use PCRE or other
> > libraries which I don't have references to. Look in the "Not an Erlang
> fan"
> > thread here which gives some references to them.
> >
> > An interesting alternative would be to write the regexp parser/compiler
> in
> > Erlang and have the basic regexp matcher in the emulator. This actually
> > wouldn't be so big, I think. Get the best of both worlds. An I used to
> > complain (still do in fact) that the emulator is to big. :-)
> >
> > Robert
> >
> > * Actually there are two types of POSIX regexps in the standard, one
> which
> > has back-references and one which doesn't.
> >
> >
> >
> > On 25/09/2007, G Bulmer <gbulmer@REDACTED> wrote:
> > > After reading the original thread, I wondered what implementation of
> > > regular expression matching is used by Erlang?
> > >
> > > I ask because this paper points out that many implementations have
> > > bad pathological behaviour:
> > > http://swtch.com/~rsc/regexp/regexp1.html
> > > An observation they make is:
> > > "Regular expression matching can be simple and fast, using finite
> > > automata-based techniques that have been known for decades. In
> > > contrast, Perl, PCRE, Python, Ruby, Java, and many other languages
> > > have regular expression implementations based on recursive
> > > backtracking that are simple but can be excruciatingly slow."
> > >
> > > This is followed by a discussion at http://lambda-the-ultimate.org/
> > > node/2064
> > > Which seems to come to a working implementation.
> > > Usefully, they note that Thompson's original patent has expired, so
> > > there is no obstacle to using the original, and apparently superior,
> > > algorithm
> > >
> > > I looked at 'man regex' on my Mac, and it describes the POSIX
> > > implementation of regex. It does contain the note "Originally written
> > > by Henry Spencer", which is suggested as the basis/culprit of the
> > > pathological regular expression implementations (I should add, as
> > > Henry Spencer is one of my heroes, that he is also credited with TCL
> > > 8 regex, which works very well according to the graphs in the paper).
> > >
> > > I have tried grep'ing through *.c in otp_src for regex and regcomp,
> > > and came up with nothing!
> > >
> > > Does anyone know what is used in Erlang? It may be a relatively small
> > > amount of work to incorporate a new regex to provide more stable
> > > behaviour.
> > >
> > > GB
> > >
> > > On 25 Sep 2007, at 10:00,
> > erlang-questions-request@REDACTED wrote:
> > >
> > > > Date: Mon, 24 Sep 2007 09:20:08 -0700 (PDT)
> > > > From: Thomas Lindgren < thomasl_erlang@REDACTED>
> > > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > > To: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
> > > > Message-ID:
> > <761781.89097.qm@REDACTED >
> > > > Content-Type: text/plain; charset=iso-8859-1
> > > >
> > > >
> > > > -
> > > > Note that this does not include the actual regexp
> > > > search of each line ... The code to do that could be
> > > > an FSM + dictionary insert instead of the simple
> > > > check-and-accumulate for newline. Extra cost? Unclear.
> > > >
> > > > One could merge that code into the scanning the binary
> > > > but this should ideally be done by a regexp compiler,
> > > > shouldn't it? Not done at this time AFAIK.
> > > >
> > > > Best,
> > > > Thomas
> > >
> > > > Date: Mon, 24 Sep 2007 19:55:12 +0200
> > > > From: Claes Wikstrom <klacke@REDACTED>
> > > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > > To: Bob Ippolito < bob@REDACTED>
> > > > Cc: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
> > > > Message-ID: < 46F7FA00.8070507@REDACTED>
> > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> > > >
> > > > Bob Ippolito wrote:
> > > >
> > > >> On 9/24/07, Patrick Logan < patrickdlogan@REDACTED> wrote:
> > > >>
> > > >>>>>>
> > http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
> > > >>>>>>
> > > >>>>>> Tim Bray might raise some valid points here, even if he's
> > > >>>>>> slightly
> > > >>>>>> biased by his background.
> > > >>>>>>
> > > >
> > > > No, the only fast way today to process a large file line/by/line is
> to
> > > >
> > > > 1. file:open(Filename, [read, raw])
> > > > 2. In a loop {ok, Bin} = file:read(Fd, BufSize),
> > > > 3. Use a binary regex matcher such as
> > > >     http://yaws.hyber.org/download/posregex-1.0.tgz
> > > >
> > > > (I don't know the state of the regex lib in OTP today, last time
> > > >   I looked it sucked bigtime though)
> > > >
> > > > /klacke
> > > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071003/eca639e3/attachment.htm>

From tblachowicz@REDACTED  Thu Oct  4 01:51:47 2007
From: tblachowicz@REDACTED (Tomasz Blachowicz)
Date: Thu, 4 Oct 2007 00:51:47 +0100
Subject: [erlang-questions] gen_tcp:connect/3 and ** exited: badarg **
Message-ID: <e53e7ea10710031651t2bce923aqb7067388fe854de7@mail.gmail.com>

Hi,

I've been having problem when I try to call gen_tcp:connect/3
function. It returns immediately with output that says: '** exited:
badarg **'.

In erlang shell it is:

1> gen_tcp:connect("www.erlang.org", 80, [binary, {pocket, 0}]).
** exited: badarg **

Any ideas?

Regards,
Tom


From erlangx@REDACTED  Thu Oct  4 01:56:23 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Wed, 3 Oct 2007 16:56:23 -0700
Subject: [erlang-questions] gen_tcp:connect/3 and ** exited: badarg **
In-Reply-To: <e53e7ea10710031651t2bce923aqb7067388fe854de7@mail.gmail.com>
References: <e53e7ea10710031651t2bce923aqb7067388fe854de7@mail.gmail.com>
Message-ID: <20071003235622.GE4892@delora.autosys.us>

On Thu, Oct 04, 2007 at 12:51:47AM +0100, Tomasz Blachowicz wrote:
> Hi,
> 
> I've been having problem when I try to call gen_tcp:connect/3
> function. It returns immediately with output that says: '** exited:
> badarg **'.
> 
> In erlang shell it is:
> 
> 1> gen_tcp:connect("www.erlang.org", 80, [binary, {pocket, 0}]).
> ** exited: badarg **
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1> gen_tcp:connect("www.erlang.org", 80, [binary, {packet, 0}]).
  {ok,#Port<0.414>}
2>

   try packet with an 'a' 


> Any ideas?
> 
> Regards,
> Tom
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,47042b3673321921969146!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From tblachowicz@REDACTED  Thu Oct  4 02:03:31 2007
From: tblachowicz@REDACTED (Tomasz Blachowicz)
Date: Thu, 4 Oct 2007 01:03:31 +0100
Subject: [erlang-questions] gen_tcp:connect/3 and ** exited: badarg **
In-Reply-To: <20071003235622.GE4892@delora.autosys.us>
References: <e53e7ea10710031651t2bce923aqb7067388fe854de7@mail.gmail.com>
	<20071003235622.GE4892@delora.autosys.us>
Message-ID: <e53e7ea10710031703o1c24e6cag4676b9be527072e9@mail.gmail.com>

Thanks, Michael! A typo... I think it's too late for hacking. Time to bed :)

On 10/4/07, Michael McDaniel <erlangx@REDACTED> wrote:
> On Thu, Oct 04, 2007 at 12:51:47AM +0100, Tomasz Blachowicz wrote:
> > Hi,
> >
> > I've been having problem when I try to call gen_tcp:connect/3
> > function. It returns immediately with output that says: '** exited:
> > badarg **'.
> >
> > In erlang shell it is:
> >
> > 1> gen_tcp:connect("www.erlang.org", 80, [binary, {pocket, 0}]).
> > ** exited: badarg **
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 1> gen_tcp:connect("www.erlang.org", 80, [binary, {packet, 0}]).
>   {ok,#Port<0.414>}
> 2>
>
>    try packet with an 'a'
>
>
> > Any ideas?
> >
> > Regards,
> > Tom
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> > !DSPAM:52,47042b3673321921969146!
> >
> >
>
> --
> Michael McDaniel
> Portland, Oregon, USA
> http://autosys.us
> +1 503 283 5284
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ok@REDACTED  Thu Oct  4 03:43:20 2007
From: ok@REDACTED (ok)
Date: Thu, 4 Oct 2007 14:43:20 +1300
Subject: [erlang-questions]
 =?iso-8859-1?q?Surprising_conclusion_in_=B7_Pa?=
 =?iso-8859-1?q?rallel_programming_environments=3A_less_is_more?=
In-Reply-To: <470349D7.2050108@ericsson.com>
References: <470349D7.2050108@ericsson.com>
Message-ID: <40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>

On 3 Oct 2007, at 8:50 pm, Bengt Kleberg wrote:
> http://blogs.intel.com/research/2007/10/ 
> parallel_programming_environme.html
> has a rather surprising conclusion to solving parallel programming
> problems. no new languages are needed, instead someone should fix the
> existing ones.

That article basically says:
   (fact) if you give people too many kinds of jam to choose from,
          they will choose less jam.  (All the examples of this kind
          presented in the article are about people's PURCHASING
          decisions.)
   (wild extrapolation) there are already too many parallel programming
	 languages and approaches.
   (extreme flight into fantasy) we should use an existing language
          rather than one designed for parallelism.

There is not, in fact, anything resembling a logical argument or any
evidence presented to link these points.  In particular, since every
parallel language is a progamming language, if there are too many
parallel languages there are too many *programming* languages of any
kind, and we should all standardise on a small number of them.  (C++?
Fortran?  Visual Basic?)  Oh, and with about 6,000 natural languages
on Earth, we should most of us be so paralysed by choice that we don't
say anything!

I've just been co-supervising a 4th year student who was comparing
MPI (C where each process has a separate address space and you do
all the parallel stuff with message passing) with TreadMarks and VODCA
(Distributed Shared Memory systems, C where some variables are shared
between processes, and communication is actually done behind your back
by catching page faults and sending differences around).  The basic
idea of DSM is that it's supposed to be a more familiar model; it's
just like programming with threads in C (with a bit of care).  He went
into it with no preconceptions, and I only got involved half way
through.  What did he find?  MPI is faster.  (Everyone finds that.)
MPI is *easier*.  A lot easier.

I have a strong suspicion that the author of that article has never
used a high level programming language (like, say, Haskell) and has
no idea just how much harder the languages he knows are making ALL
his programming, not just the parallel part.

We've had axes for 500,000 years and people are still injuring  
themselves
with them.  We've had computers for about 50.  We have a long way to go
before we know what we're doing.





From mark-erlang@REDACTED  Thu Oct  4 05:26:22 2007
From: mark-erlang@REDACTED (Mark Aufflick)
Date: Thu, 4 Oct 2007 13:26:22 +1000
Subject: [erlang-questions] Using Lisp to extend a Mac IDE for Erlang
In-Reply-To: <AB032F81-4F53-4ECD-9E8C-B63AE79C04A6@gmail.com>
References: <AB032F81-4F53-4ECD-9E8C-B63AE79C04A6@gmail.com>
Message-ID: <1242f34a0710032026r6e943808wb5ac3273155ebf54@mail.gmail.com>

Sounds like emacs to me :)


On 10/4/07, Joel Reymont <joelr1@REDACTED> wrote:
> I'm between projects right now (anyone want to hire me?) so I have
> time to spare. I'm once again thinking of the Mac IDE for Erlang.
> More specifically, I'm thinking of the language used to extend such
> an IDE.
>
> I'm wondering if there are any objections to an environment that
> reuses the Distel code and can be extended in Lisp. Sort of like a
> TextMate for Erlang with Lisp instead of Ruby.
>
> I already own a LispWorks license and it comes with a Cocoa bridge so
> the only thing I would need to do to bootstrap is port Distel and
> hook it up to an editor. What do you think?
>
>         Thanks, Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Mark Aufflick
  contact info at http://mark.aufflick.com/about/contact


From davidnwelton@REDACTED  Thu Oct  4 10:43:12 2007
From: davidnwelton@REDACTED (David Welton)
Date: Thu, 4 Oct 2007 10:43:12 +0200
Subject: [erlang-questions]
	=?iso-8859-1?q?Surprising_conclusion_in_=B7_Pa?=
	=?iso-8859-1?q?rallel_programming_environments=3A_less_is_more?=
In-Reply-To: <40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
References: <470349D7.2050108@ericsson.com>
	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
Message-ID: <9877cd600710040143j573ccec1ydbe3be9a2b1f9065@mail.gmail.com>

> > http://blogs.intel.com/research/2007/10/
> > parallel_programming_environme.html
> > has a rather surprising conclusion to solving parallel programming
> > problems. no new languages are needed, instead someone should fix the
> > existing ones.
>
> That article basically says:
>    (fact) if you give people too many kinds of jam to choose from,
>           they will choose less jam.  (All the examples of this kind
>           presented in the article are about people's PURCHASING
>           decisions.)
>    (wild extrapolation) there are already too many parallel programming
>          languages and approaches.
>    (extreme flight into fantasy) we should use an existing language
>           rather than one designed for parallelism.
>
> There is not, in fact, anything resembling a logical argument or any
> evidence presented to link these points.  In particular, since every
> parallel language is a progamming language, if there are too many
> parallel languages there are too many *programming* languages of any
> kind, and we should all standardise on a small number of them.  (C++?
> Fortran?  Visual Basic?)  Oh, and with about 6,000 natural languages
> on Earth, we should most of us be so paralysed by choice that we don't
> say anything!

The Paradox of Choice book is pretty good, and while the above
conclusions are off, the concept is valuable.

Think of how anxious some people get trying to pick the "best"
programming language, especially because as someone with less
experience, they have less skills to even judge.

Of course, it's not possible to constrain the amount of programming
languages - the market does an ok job of doing that, but for a
practical application of this concept, consider Object Oriented
extensions to the Tcl programming language.  Tcl is a flexible enough
language that you can add OO systems on top of the base language (even
writing them in Tcl itself, or going through the C extension system).
Of course, the first thing most people ask, when they decide to add OO
to their Tcl is "which one should I use?  which one is best?".  With
something like Python or Ruby, this doubt simply does not come into
play.  I think that in the end, having to choose is more of a burden
than simply getting a good default (especially when you consider the
negative effects of not being able to share code between all the
different OO systems).

So, yeah, their example is bad, but the concept is worth paying attention to.

-- 
David N. Welton
http://www.welton.it/davidw/


From bbmaj7@REDACTED  Thu Oct  4 12:37:34 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Thu, 4 Oct 2007 20:37:34 +1000 (EST)
Subject: [erlang-questions] learning mnesia:select syntax
Message-ID: <471441.99048.qm@web52011.mail.re2.yahoo.com>

I've been studying mnesia select (and ERTS by extension) from OTP docs for days
and I am baffled. Anyone know of good material for learning this?
Is it worth looking at the mnesia code?



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From launoja@REDACTED  Thu Oct  4 15:47:25 2007
From: launoja@REDACTED (Jani Launonen)
Date: Thu, 4 Oct 2007 16:47:25 +0300
Subject: [erlang-questions] learning mnesia:select syntax
In-Reply-To: <471441.99048.qm@web52011.mail.re2.yahoo.com>
References: <471441.99048.qm@web52011.mail.re2.yahoo.com>
Message-ID: <DFB8E030-7F5F-4D43-A591-FB2EA5584A96@omanetti.fi>


Richard Andrews kirjoitti 4.10.2007 kello 13.37:

> I've been studying mnesia select (and ERTS by extension) from OTP  
> docs for days
> and I am baffled. Anyone know of good material for learning this?
> Is it worth looking at the mnesia code?

I think that mnesia documentation is quite nice. Perhaps you should  
have some hands on experimenting to become more used to the select?

I've never used match_spec in my life, but I started an erlang shell,  
cut'n'pasted from http://www.erlang.org/doc/apps/mnesia/ 
Mnesia_chap5.html#5 chapter 5.10 the example tables fruit and  
vegetable to a file (fruits.txt) and loaded it from the shell by

mnesia:load_textfile("fruits.txt").

I defined the same records for shell just as a convenience

rd(fruit, {name, color, taste}).
rd(vegetable, {name, color, taste, price}).

and after some tinkering made match_specs from select example in  
mnesia documentation:
MatchHead = #fruit{name='$1', _='_', taste=sweet}.
Guards = [].
Results = ['$1'].

Then the selection:
mnesia:dirty_select(fruit, [{MatchHead, Guards, Results}]).

which returned [orange, apple] as expected.

The same with vegetable table:
MatchHead1 = #vegetable{name='$1', _='_', _='_', price='$2'}.
Guards1 = [{'>', 2.0, '$2'}].

mnesia:dirty_select(vegetable, [{MatchHead1, Guards1, Results}]).

which resulted in [potato] as expected.

Just try these and try making more compicated examples stepwisely. I  
hope that helped.

Cheers,
Jani Launonen


>
>
>
>       Sick of deleting your inbox? Yahoo!7 Mail has free unlimited  
> storage.
> http://au.docs.yahoo.com/mail/unlimitedstorage.html
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From dmercer@REDACTED  Thu Oct  4 19:41:09 2007
From: dmercer@REDACTED (David Mercer)
Date: Thu, 4 Oct 2007 12:41:09 -0500
Subject: [erlang-questions] Surprising conclusion in . Parallel
	programming environments: less is more
In-Reply-To: <40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
References: <470349D7.2050108@ericsson.com>
	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
Message-ID: <015e01c806ad$bff25de0$891ea8c0@SSI.CORP>

Ok:
> Oh, and with about 6,000 natural languages
> on Earth, we should most of us be so paralysed by choice that we don't
> say anything!

Good point.  I need to go check out some of those other languages to see if
they would actually be better than English.  Anyone got any suggestions?

I've heard Chinese is really good, because one character can convey an
entire word, rather than having to string multiple characters together.

On the other hand, the smaller alphabet of English is kind of like a macro
capability, since you can form new words out of the basic building blocks.
Maybe a language with a smaller alphabet is better though.  Greek has only
24 letters...

Cheers,

David



From dmercer@REDACTED  Thu Oct  4 19:43:40 2007
From: dmercer@REDACTED (David Mercer)
Date: Thu, 4 Oct 2007 12:43:40 -0500
Subject: [erlang-questions] Regular expression library (was Not anErlang
	fan)
In-Reply-To: <3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org><62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com><3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com><17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
	<3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>
Message-ID: <015f01c806ae$19eba950$891ea8c0@SSI.CORP>

How do you create a module on the fly?  Wouldn't you have to write it to
disk and then compile and load it?  Or is there another way?

 

Cheers,

 

David

 

  _____  

From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Robert Virding
Sent: Wednesday, October 03, 2007 16:32
To: Yariv Sadan
Cc: erlang-questions@REDACTED; G Bulmer
Subject: Re: [erlang-questions] Regular expression library (was Not anErlang
fan)

 

That's not too difficult to do at all. The speed you get would depend on
which regexp family you do it for.

Robert

On 27/09/2007, Yariv Sadan <yarivsadan@REDACTED> wrote:

I was wondering if it would be possible/beneficial to write a regexp 
compiler that takes a regexp string and compiles it into an Erlang
module.

You would use it as follows:

regexp2:compile("f*o+o?", myregexp, [native]),
true = myregexp:matches("foo"). 

This could be a very fast regexp engine.

What do you think?

Yariv

On 9/26/07, Robert Virding <rvirding@REDACTED> wrote:
> The regular expression module which is included in the distribution is a 
> simple rather na?ve one written in Erlang which uses a back-tracking
> algorithm directly on the AST. It has all the problems Russ Cox mentioned
in
> his paper. I wrote it long ago when no one was interested in parsing
regexps 
> in Erlang. :-)
>
> I am rewriting it at the moment, still in Erlang but using the same
> algorithm, or similar to the one, which Russ describes. It is about the
same
> speed as the existing one for simpler regexps but it can handle all the 
> pathological cases as it should. It will handle POSIX syntax and semantics
> (but no back-references*). There will also be a version which will
implement
> a subset of PERL regexps with PERL semantics. Both will work on either
lists 
> or binaries.
>
> I hadn't seen the lambda-the-ultimate discussion you mentioned and will
> check it out.
>
> There are a number of regexp packages for Erlang which use PCRE or other 
> libraries which I don't have references to. Look in the "Not an Erlang
fan"
> thread here which gives some references to them.
>
> An interesting alternative would be to write the regexp parser/compiler in

> Erlang and have the basic regexp matcher in the emulator. This actually
> wouldn't be so big, I think. Get the best of both worlds. An I used to
> complain (still do in fact) that the emulator is to big. :-) 
>
> Robert
>
> * Actually there are two types of POSIX regexps in the standard, one which
> has back-references and one which doesn't.
>
>
>
> On 25/09/2007, G Bulmer < gbulmer@REDACTED> wrote:
> > After reading the original thread, I wondered what implementation of
> > regular expression matching is used by Erlang?
> > 
> > I ask because this paper points out that many implementations have
> > bad pathological behaviour:
> > http://swtch.com/~rsc/regexp/regexp1.html 
> > An observation they make is:
> > "Regular expression matching can be simple and fast, using finite
> > automata-based techniques that have been known for decades. In
> > contrast, Perl, PCRE, Python, Ruby, Java, and many other languages 
> > have regular expression implementations based on recursive
> > backtracking that are simple but can be excruciatingly slow."
> >
> > This is followed by a discussion at http://lambda-the-ultimate.org/
> > node/2064
> > Which seems to come to a working implementation.
> > Usefully, they note that Thompson's original patent has expired, so
> > there is no obstacle to using the original, and apparently superior, 
> > algorithm
> >
> > I looked at 'man regex' on my Mac, and it describes the POSIX
> > implementation of regex. It does contain the note "Originally written
> > by Henry Spencer", which is suggested as the basis/culprit of the 
> > pathological regular expression implementations (I should add, as
> > Henry Spencer is one of my heroes, that he is also credited with TCL
> > 8 regex, which works very well according to the graphs in the paper). 
> >
> > I have tried grep'ing through *.c in otp_src for regex and regcomp,
> > and came up with nothing!
> >
> > Does anyone know what is used in Erlang? It may be a relatively small 
> > amount of work to incorporate a new regex to provide more stable
> > behaviour.
> >
> > GB
> >
> > On 25 Sep 2007, at 10:00,
> erlang-questions-request@REDACTED wrote:
> >
> > > Date: Mon, 24 Sep 2007 09:20:08 -0700 (PDT)
> > > From: Thomas Lindgren < thomasl_erlang@REDACTED
<mailto:thomasl_erlang@REDACTED> >
> > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > To: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED> 
> > > Message-ID:
> <761781.89097.qm@REDACTED >
> > > Content-Type: text/plain; charset=iso-8859-1
> > >
> > >
> > > -
> > > Note that this does not include the actual regexp
> > > search of each line ... The code to do that could be
> > > an FSM + dictionary insert instead of the simple 
> > > check-and-accumulate for newline. Extra cost? Unclear.
> > >
> > > One could merge that code into the scanning the binary
> > > but this should ideally be done by a regexp compiler, 
> > > shouldn't it? Not done at this time AFAIK.
> > >
> > > Best,
> > > Thomas
> >
> > > Date: Mon, 24 Sep 2007 19:55:12 +0200
> > > From: Claes Wikstrom < klacke@REDACTED>
> > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > To: Bob Ippolito < bob@REDACTED> 
> > > Cc: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
> > > Message-ID: < 46F7FA00.8070507@REDACTED>
> > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> > >
> > > Bob Ippolito wrote:
> > >
> > >> On 9/24/07, Patrick Logan < patrickdlogan@REDACTED> wrote:
> > >>
> > >>>>>>
> http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang 
> > >>>>>>
> > >>>>>> Tim Bray might raise some valid points here, even if he's
> > >>>>>> slightly
> > >>>>>> biased by his background. 
> > >>>>>>
> > >
> > > No, the only fast way today to process a large file line/by/line is to
> > >
> > > 1. file:open(Filename, [read, raw])
> > > 2. In a loop {ok, Bin} = file:read(Fd, BufSize),
> > > 3. Use a binary regex matcher such as
> > >     http://yaws.hyber.org/download/posregex-1.0.tgz 
> > >
> > > (I don't know the state of the regex lib in OTP today, last time
> > >   I looked it sucked bigtime though)
> > >
> > > /klacke
> > > 
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071004/4aa2f40c/attachment.htm>

From cyberlync@REDACTED  Thu Oct  4 19:54:46 2007
From: cyberlync@REDACTED (Eric Merritt)
Date: Thu, 4 Oct 2007 10:54:46 -0700
Subject: [erlang-questions] Erlang release management
In-Reply-To: <9D1F5713-B4DE-4866-87A4-1092CA27AFAF@process-one.net>
References: <d791b8790709271221s45b5aa58qe230c85e359f7b85@mail.gmail.com>
	<m1ir5v3iy1.fsf@aegir.wgtn.cat-it.co.nz>
	<4B3109BF-2CE8-4768-A996-AC3188F1AF5C@process-one.net>
	<d791b8790709280730h5487328epf3ca89beb283b316@mail.gmail.com>
	<f69c840c0709281710n2e7bdedfkaedaeafd3888d7e4@mail.gmail.com>
	<9D1F5713-B4DE-4866-87A4-1092CA27AFAF@process-one.net>
Message-ID: <f69c840c0710041054v455c1b7cn216f4ecd9ce53abf@mail.gmail.com>

>
> Eric, may sinan be able, at build stage, to generate some kind of
> generic code reload mecanism developer can plug its application loop
> on ?
>

If such a mechanism exists it should be pretty easy to plug into
sinan, its what it was designed for.  In fact, I already generate
quite a bit of the OTP stuff (*.rel, *.boot, *.script, etc) so adding
a bit of additional stuff shouldn't be a problem.  What did you have
in mind?


From qrilka@REDACTED  Thu Oct  4 20:22:49 2007
From: qrilka@REDACTED (Kirill Zaborski)
Date: Thu, 4 Oct 2007 22:22:49 +0400
Subject: [erlang-questions] Regular expression library (was Not anErlang
	fan)
In-Reply-To: <015f01c806ae$19eba950$891ea8c0@SSI.CORP>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
	<3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>
	<015f01c806ae$19eba950$891ea8c0@SSI.CORP>
Message-ID: <337538cb0710041122l41b68493o75e02164db46bd40@mail.gmail.com>

Maybe with code:load_binary/3 ?

Best regards,
Kirill

On 10/4/07, David Mercer <dmercer@REDACTED> wrote:
>
>  How do you create a module on the fly?  Wouldn't you have to write it to
> disk and then compile and load it?  Or is there another way?
>
>
>
> Cheers,
>
>
>
> David
>
>
>  ------------------------------
>
> *From:* erlang-questions-bounces@REDACTED [mailto:
> erlang-questions-bounces@REDACTED] *On Behalf Of *Robert Virding
> *Sent:* Wednesday, October 03, 2007 16:32
> *To:* Yariv Sadan
> *Cc:* erlang-questions@REDACTED; G Bulmer
> *Subject:* Re: [erlang-questions] Regular expression library (was Not
> anErlang fan)
>
>
>
> That's not too difficult to do at all. The speed you get would depend on
> which regexp family you do it for.
>
> Robert
>
> On 27/09/2007, *Yariv Sadan* <yarivsadan@REDACTED> wrote:
>
> I was wondering if it would be possible/beneficial to write a regexp
> compiler that takes a regexp string and compiles it into an Erlang
> module.
>
> You would use it as follows:
>
> regexp2:compile("f*o+o?", myregexp, [native]),
> true = myregexp:matches("foo").
>
> This could be a very fast regexp engine.
>
> What do you think?
>
> Yariv
>
> On 9/26/07, Robert Virding <rvirding@REDACTED> wrote:
> > The regular expression module which is included in the distribution is a
>
> > simple rather na?ve one written in Erlang which uses a back-tracking
> > algorithm directly on the AST. It has all the problems Russ Cox
> mentioned in
> > his paper. I wrote it long ago when no one was interested in parsing
> regexps
> > in Erlang. :-)
> >
> > I am rewriting it at the moment, still in Erlang but using the same
> > algorithm, or similar to the one, which Russ describes. It is about the
> same
> > speed as the existing one for simpler regexps but it can handle all the
> > pathological cases as it should. It will handle POSIX syntax and
> semantics
> > (but no back-references*). There will also be a version which will
> implement
> > a subset of PERL regexps with PERL semantics. Both will work on either
> lists
> > or binaries.
> >
> > I hadn't seen the lambda-the-ultimate discussion you mentioned and will
> > check it out.
> >
> > There are a number of regexp packages for Erlang which use PCRE or other
>
> > libraries which I don't have references to. Look in the "Not an Erlang
> fan"
> > thread here which gives some references to them.
> >
> > An interesting alternative would be to write the regexp parser/compiler
> in
> > Erlang and have the basic regexp matcher in the emulator. This actually
> > wouldn't be so big, I think. Get the best of both worlds. An I used to
> > complain (still do in fact) that the emulator is to big. :-)
> >
> > Robert
> >
> > * Actually there are two types of POSIX regexps in the standard, one
> which
> > has back-references and one which doesn't.
> >
> >
> >
> > On 25/09/2007, G Bulmer < gbulmer@REDACTED> wrote:
> > > After reading the original thread, I wondered what implementation of
> > > regular expression matching is used by Erlang?
> > >
> > > I ask because this paper points out that many implementations have
> > > bad pathological behaviour:
> > > http://swtch.com/~rsc/regexp/regexp1.html
> <http://swtch.com/%7Ersc/regexp/regexp1.html>
> > > An observation they make is:
> > > "Regular expression matching can be simple and fast, using finite
> > > automata-based techniques that have been known for decades. In
> > > contrast, Perl, PCRE, Python, Ruby, Java, and many other languages
> > > have regular expression implementations based on recursive
> > > backtracking that are simple but can be excruciatingly slow."
> > >
> > > This is followed by a discussion at http://lambda-the-ultimate.org/
> > > node/2064
> > > Which seems to come to a working implementation.
> > > Usefully, they note that Thompson's original patent has expired, so
> > > there is no obstacle to using the original, and apparently superior,
> > > algorithm
> > >
> > > I looked at 'man regex' on my Mac, and it describes the POSIX
> > > implementation of regex. It does contain the note "Originally written
> > > by Henry Spencer", which is suggested as the basis/culprit of the
> > > pathological regular expression implementations (I should add, as
> > > Henry Spencer is one of my heroes, that he is also credited with TCL
> > > 8 regex, which works very well according to the graphs in the paper).
> > >
> > > I have tried grep'ing through *.c in otp_src for regex and regcomp,
> > > and came up with nothing!
> > >
> > > Does anyone know what is used in Erlang? It may be a relatively small
> > > amount of work to incorporate a new regex to provide more stable
> > > behaviour.
> > >
> > > GB
> > >
> > > On 25 Sep 2007, at 10:00,
> > erlang-questions-request@REDACTED wrote:
> > >
> > > > Date: Mon, 24 Sep 2007 09:20:08 -0700 (PDT)
> > > > From: Thomas Lindgren < thomasl_erlang@REDACTED >
> > > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > > To: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
> > > > Message-ID:
> > <761781.89097.qm@REDACTED >
> > > > Content-Type: text/plain; charset=iso-8859-1
> > > >
> > > >
> > > > -
> > > > Note that this does not include the actual regexp
> > > > search of each line ... The code to do that could be
> > > > an FSM + dictionary insert instead of the simple
> > > > check-and-accumulate for newline. Extra cost? Unclear.
> > > >
> > > > One could merge that code into the scanning the binary
> > > > but this should ideally be done by a regexp compiler,
> > > > shouldn't it? Not done at this time AFAIK.
> > > >
> > > > Best,
> > > > Thomas
> > >
> > > > Date: Mon, 24 Sep 2007 19:55:12 +0200
> > > > From: Claes Wikstrom < klacke@REDACTED>
> > > > Subject: Re: [erlang-questions] Not an Erlang fan
> > > > To: Bob Ippolito < bob@REDACTED>
> > > > Cc: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
> > > > Message-ID: < 46F7FA00.8070507@REDACTED>
> > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> > > >
> > > > Bob Ippolito wrote:
> > > >
> > > >> On 9/24/07, Patrick Logan < patrickdlogan@REDACTED> wrote:
> > > >>
> > > >>>>>>
> > http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
> > > >>>>>>
> > > >>>>>> Tim Bray might raise some valid points here, even if he's
> > > >>>>>> slightly
> > > >>>>>> biased by his background.
> > > >>>>>>
> > > >
> > > > No, the only fast way today to process a large file line/by/line is
> to
> > > >
> > > > 1. file:open(Filename, [read, raw])
> > > > 2. In a loop {ok, Bin} = file:read(Fd, BufSize),
> > > > 3. Use a binary regex matcher such as
> > > >     http://yaws.hyber.org/download/posregex-1.0.tgz
> > > >
> > > > (I don't know the state of the regex lib in OTP today, last time
> > > >   I looked it sucked bigtime though)
> > > >
> > > > /klacke
> > > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071004/2c24fb6f/attachment.htm>

From garry@REDACTED  Thu Oct  4 20:30:54 2007
From: garry@REDACTED (Garry Hodgson)
Date: Thu, 4 Oct 2007 14:30:54 -0400 (EDT)
Subject: [erlang-questions] Surprising conclusion in Parallel
	programming environments: less	is more?
In-Reply-To: <40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
References: <40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
Message-ID: <2007100414301191522654@k2.sage.att.com>

ok <ok@REDACTED> wrote:

> That article basically says:
>    (fact) if you give people too many kinds of jam to choose from,
>           they will choose less jam.  (All the examples of this kind
>           presented in the article are about people's PURCHASING
>           decisions.)
>    (wild extrapolation) there are already too many parallel programming
>   languages and approaches.
>    (extreme flight into fantasy) we should use an existing language
>           rather than one designed for parallelism.

if, indeed, there are too many languages, it's not because we make
up new ones, but because we never retire old ones.

----
Garry Hodgson, Senior Software Geek, AT&T CSO

nobody can do everything, but everybody can do something.
do something.



From garry@REDACTED  Thu Oct  4 21:00:58 2007
From: garry@REDACTED (Garry Hodgson)
Date: Thu, 4 Oct 2007 15:00:58 -0400 (EDT)
Subject: [erlang-questions] logging to syslog
Message-ID: <2007100415001191524458@k2.sage.att.com>

i'm trying to use the syslog package from the user contributions section of 
the erlang website to log errors to syslog.  the module compiles and runs fine, 
produces no complaints, but i never see any of its log messages show up in 
the system log files (/var/log/messages, in my case).  

i've looked through the code, and read the RFC, and i have every reason to 
believe that the code is correct.  especially since i can't seem to write to 
syslog using netcat to send example messages from the RFC, either.  the 
logger(1) command (using either unix stream or datagram methods), and 
python's syslog module both do work, so syslogd appears to be sane.

so i'm wondering if there is some subtlety in usage or configuration that 
i'm missing here?  does anybody have any tips?

thanks


----
Garry Hodgson, Senior Software Geek, AT&T CSO

nobody can do everything, but everybody can do something.
do something.



From stephan@REDACTED  Thu Oct  4 21:10:41 2007
From: stephan@REDACTED (Stephan Maka)
Date: Thu, 4 Oct 2007 21:10:41 +0200
Subject: [erlang-questions] gen_tcp and processes receiving
Message-ID: <20071004191041.GA17261@chronos.sin>

Hi

I am having trouble receiving from TCP sockets in processes that did not
open it. The processes are not distributed over multiple nodes but `erl'
is invoked with `-smp'.

The attached module simply received nothing. When I receive in the
process that opened the socket it works. In the program where I need
this functionality the gen_tcp:recv result is {error, ebadf}. I did some
googling on it, but all people with the same problem have gotten no
answer to it.

I'm unsure about the use of gen_tcp:controlling_process when I simply
use a passive socket. Active sockets are no option here, I have strong
traffic shaping requirements.

Thanks for any tips in advance,
Stephan
-------------- next part --------------
-module(tcp_ebadf).

-export([start/0]).


start() ->
    io:format("Connecting...~n"),
    {ok, Socket} = gen_tcp:connect("www.erlang.org", 80, [{active, false}]),

    io:format("Sending...~n"),
    gen_tcp:send(Socket, "GET / HTTP/1.1\r\n"
		 "Host: www.erlang.org\r\n"
		 "Connection: close\r\n"
		 "\r\n"),

    Pid = spawn_link(fun() ->
			     receive
				 go ->
				     ok
			     end,
			     receiver(self(), Socket)
		     end),
    ok = gen_tcp:controlling_process(Socket, Pid),
    Pid ! go,

    io:format("Waiting...~n"),
    receive
	{byte, _B} ->
	    io:format("Ok, I received a byte~n")
    after 10000 ->
	    exit(timeout)
    end,
    
    gen_tcp:close(Socket).


receiver(Listener, Socket) ->
    io:format("Receiving...~n"),
    {ok, [B]} = gen_tcp:recv(Socket, 1),
    Listener ! {byte, B}.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071004/a9c27cc9/attachment.bin>

From micha-1@REDACTED  Thu Oct  4 22:11:10 2007
From: micha-1@REDACTED (Michael Wohlwend)
Date: Thu, 4 Oct 2007 22:11:10 +0200
Subject: [erlang-questions] logging to syslog
In-Reply-To: <2007100415001191524458@k2.sage.att.com>
References: <2007100415001191524458@k2.sage.att.com>
Message-ID: <200710042211.10986.micha-1@fantasymail.de>

Am Donnerstag 04 Oktober 2007 21:00:58 schrieb Garry Hodgson:
> i'm trying to use the syslog package from the user contributions section of
> the erlang website to log errors to syslog.  the module compiles and runs
> fine, produces no complaints, but i never see any of its log messages show
> up in the system log files (/var/log/messages, in my case).

syslogd has to be started with option "-r" to make it listen on the udp port.
just change it in /etc/default/syslogd

cheers
 Michael


From per@REDACTED  Thu Oct  4 22:37:43 2007
From: per@REDACTED (Per Hedeland)
Date: Thu, 4 Oct 2007 22:37:43 +0200 (CEST)
Subject: [erlang-questions] logging to syslog
In-Reply-To: <2007100415001191524458@k2.sage.att.com>
Message-ID: <200710042037.l94KbhgM001049@pluto.hedeland.org>

Garry Hodgson <garry@REDACTED> wrote:
>
>i've looked through the code, and read the RFC, and i have every reason to 
>believe that the code is correct.  especially since i can't seem to write to 
>syslog using netcat to send example messages from the RFC, either.  the 
>logger(1) command (using either unix stream or datagram methods), and 
>python's syslog module both do work, so syslogd appears to be sane.

Have you verified that logger is actually using the UDP port, and
consequently, that your syslog server is actually listening on it?
Basically none do it by default these days.

FWIW, the module works fine for me (on FreeBSD 6.2) after telling the
syslog daemon to listen on the port - but before doing that, 'logger -h
<myhostname> ...' did not work either. Incidentally, I don't see a way
to tell the logger version found on a nearby Linux box to use UDP... -
Hm, "datagram" /= "UDP", maybe that's where your mistake is?

--Per Hedeland


From per@REDACTED  Thu Oct  4 22:44:40 2007
From: per@REDACTED (Per Hedeland)
Date: Thu, 4 Oct 2007 22:44:40 +0200 (CEST)
Subject: [erlang-questions] logging to syslog
In-Reply-To: <200710042211.10986.micha-1@fantasymail.de>
Message-ID: <200710042044.l94KieXL001116@pluto.hedeland.org>

Michael Wohlwend <micha-1@REDACTED> wrote:
>
>Am Donnerstag 04 Oktober 2007 21:00:58 schrieb Garry Hodgson:
>> i'm trying to use the syslog package from the user contributions section of
>> the erlang website to log errors to syslog.  the module compiles and runs
>> fine, produces no complaints, but i never see any of its log messages show
>> up in the system log files (/var/log/messages, in my case).
>
>syslogd has to be started with option "-r" to make it listen on the udp port.
>just change it in /etc/default/syslogd

Ehrm, assuming you run Linux (or "a Linux distribution" for the
nitpickers:-), that is (on both counts).

--Per



From garry@REDACTED  Thu Oct  4 23:07:06 2007
From: garry@REDACTED (Garry Hodgson)
Date: Thu, 4 Oct 2007 17:07:06 -0400 (EDT)
Subject: [erlang-questions] thanks to all for the syslog help
In-Reply-To: <200710042211.10986.micha-1@fantasymail.de>
References: <200710042211.10986.micha-1@fantasymail.de>
Message-ID: <2007100417071191532026@k2.sage.att.com>

Michael Wohlwend <micha-1@REDACTED> wrote:

> syslogd has to be started with option "-r" to make it listen on the udp port.
> just change it in /etc/default/syslogd

aha!  that was the problem.  many thanks to all who responded.
this list is great.

----
Garry Hodgson, Senior Software Geek, AT&T CSO

nobody can do everything, but everybody can do something.
do something.



From 0x6e6562@REDACTED  Thu Oct  4 23:07:01 2007
From: 0x6e6562@REDACTED (Ben Hood)
Date: Thu, 4 Oct 2007 22:07:01 +0100
Subject: [erlang-questions] Dynamically dispatching encoded messages to
	functions
Message-ID: <269388e30710041407s68e49cc6k768655ec7631b3d0@mail.gmail.com>

Hi,

I'm trying to implement a generalized message handling mechanism for
messages that are send to AMQP subscribers and would like some advice
about a clean Erlang way to achieve this.

An AMQP subscriber receives messages in the following pattern:

{content, Properties, Payload}

whereby both terms are opaque binaries that must get decoded to Erlang terms.

I have functions that would decode these binaries and re-encode a
response, but I don't want to put that plumbing into modules that
implement business logic.

The function I have decodes the payload into a [ FunctionName |
Arguments ] head/tail, so that I can dynamically dispatch the
arguments to a specific function.

Ideally I would like to implement specific business logic in a
transport agnostic fashion, for example I write simple gen_server
behaviours that are not coupled with the AMQP plumbing.

I have thought of a few approaches to this problem:

1. Decode the payload in a generalized gen_event instance and look up
the target module and function

a) using a apply(M,F,A) pattern
b) or creating Funs that are keyed on the FunctionName parameter.

2. Create a gen_server template as an include, so that the plumbing
code and non-business logic specific callbacks do not need to be
repeated.

3. Implement all entry points to every business function in the same
module and use one gen_server instance to handle the communication
plumbing.

4. Just repeat myself in every business specific gen_server instance,
minimizing this by extracting as much plumbing into some external
module.

Each approach has it's pro's and con's:

1. Neat, but the dynamic dispatch may be expensive.

2. No repetitive source code, but having statically linked functions
(although quick) may prove to be a maintenance problem (I think, maybe
because of code reloading)

3. Easy to implement but this becomes very monolithic very quickly and
the gen_server state is a kitchen sink.

4. Seems least intrusive of all but I think there will be a degree of
code repetition here, especially for the function signatures.

Does anybody have any experience or known of any examples where this
type of problem is solved in Erlang, or am I completely missing the
point?

Thx,

Ben


From yinso.chen@REDACTED  Fri Oct  5 00:04:19 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 4 Oct 2007 15:04:19 -0700
Subject: [erlang-questions] parameterized module returns badfun
Message-ID: <779bf2730710041504m2a0714b9w3068920abe99c987@mail.gmail.com>

Hi -

I tried out the parameterized module example in
http://www.erlang.org/pipermail/erlang-questions/2007-May/026867.html - I
can compile the example and even create the module instance, but evaluating
the functions returns badfun instead.  I'm not sure where the issues are -
any pointers are appreciated.  I'm using 5.5.5 on linux.

Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe]
[kernel-poll:false]

Eshell V5.5.5  (abort with ^G)
1> c(pmod).
{ok,pmod}
2> M = pmod:new(3,4).
{pmod,3,4}
3> M:sum().

=ERROR REPORT==== 2-Oct-2007::10:04:26 ===
Error in process <0.30.0> with exit value:
{{badfun,{{pmod,3,4},sum}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}

** exited: {{badfun,{{pmod,3,4},sum}},
            [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]} **
4> M:new_Y(5).

=ERROR REPORT==== 2-Oct-2007::10:09:13 ===
Error in process <0.38.0> with exit value:
{{badfun,{{pmod,3,4},new_Y}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}

** exited: {{badfun,{{pmod,3,4},new_Y}},
            [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]} **
5>

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071004/40c599bf/attachment.htm>

From yarivsadan@REDACTED  Thu Oct  4 20:24:19 2007
From: yarivsadan@REDACTED (Yariv Sadan)
Date: Thu, 4 Oct 2007 11:24:19 -0700
Subject: [erlang-questions] Regular expression library (was Not anErlang
	fan)
In-Reply-To: <015f01c806ae$19eba950$891ea8c0@SSI.CORP>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
	<3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>
	<015f01c806ae$19eba950$891ea8c0@SSI.CORP>
Message-ID: <17244f480710041124s4dff2b8rf35bec8d2e7976e8@mail.gmail.com>

You can create its abstract forms and then compile them into a .beam
binary and load it into the VM at runtime. ErlyWeb uses this
extensively. Check out smerl.erl (http://code.google.com/p/erlyweb).

Yariv

On 10/4/07, David Mercer <dmercer@REDACTED> wrote:
>
>
>
>
> How do you create a module on the fly?  Wouldn't you have to write it to
> disk and then compile and load it?  Or is there another way?
>
>
>
> Cheers,
>
>
>
> David
>
>
>
>  ________________________________
>
>
> From: erlang-questions-bounces@REDACTED
> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of
> Robert Virding
>  Sent: Wednesday, October 03, 2007 16:32
>  To: Yariv Sadan
>  Cc: erlang-questions@REDACTED; G Bulmer
>  Subject: Re: [erlang-questions] Regular expression library (was Not
> anErlang fan)
>
>
>
>
> That's not too difficult to do at all. The speed you get would depend on
> which regexp family you do it for.
>
>  Robert
>
>
> On 27/09/2007, Yariv Sadan <yarivsadan@REDACTED> wrote:
>
> I was wondering if it would be possible/beneficial to write a regexp
>  compiler that takes a regexp string and compiles it into an Erlang
>  module.
>
>  You would use it as follows:
>
>  regexp2:compile("f*o+o?", myregexp, [native]),
>  true = myregexp:matches("foo").
>
>  This could be a very fast regexp engine.
>
>  What do you think?
>
>  Yariv
>
>  On 9/26/07, Robert Virding <rvirding@REDACTED> wrote:
>  > The regular expression module which is included in the distribution is a
>  > simple rather na?ve one written in Erlang which uses a back-tracking
>  > algorithm directly on the AST. It has all the problems Russ Cox mentioned
> in
>  > his paper. I wrote it long ago when no one was interested in parsing
> regexps
>  > in Erlang. :-)
>  >
>  > I am rewriting it at the moment, still in Erlang but using the same
>  > algorithm, or similar to the one, which Russ describes. It is about the
> same
>  > speed as the existing one for simpler regexps but it can handle all the
>  > pathological cases as it should. It will handle POSIX syntax and
> semantics
>  > (but no back-references*). There will also be a version which will
> implement
>  > a subset of PERL regexps with PERL semantics. Both will work on either
> lists
>  > or binaries.
>  >
>  > I hadn't seen the lambda-the-ultimate discussion you mentioned and will
>  > check it out.
>  >
>  > There are a number of regexp packages for Erlang which use PCRE or other
>  > libraries which I don't have references to. Look in the "Not an Erlang
> fan"
>  > thread here which gives some references to them.
>  >
>  > An interesting alternative would be to write the regexp parser/compiler
> in
>  > Erlang and have the basic regexp matcher in the emulator. This actually
>  > wouldn't be so big, I think. Get the best of both worlds. An I used to
>  > complain (still do in fact) that the emulator is to big. :-)
>  >
>  > Robert
>  >
>  > * Actually there are two types of POSIX regexps in the standard, one
> which
>  > has back-references and one which doesn't.
>  >
>  >
>  >
>  > On 25/09/2007, G Bulmer < gbulmer@REDACTED> wrote:
>  > > After reading the original thread, I wondered what implementation of
>  > > regular expression matching is used by Erlang?
>  > >
>  > > I ask because this paper points out that many implementations have
>  > > bad pathological behaviour:
>  > > http://swtch.com/~rsc/regexp/regexp1.html
>  > > An observation they make is:
>  > > "Regular expression matching can be simple and fast, using finite
>  > > automata-based techniques that have been known for decades. In
>  > > contrast, Perl, PCRE, Python, Ruby, Java, and many other languages
>  > > have regular expression implementations based on recursive
>  > > backtracking that are simple but can be excruciatingly slow."
>  > >
>  > > This is followed by a discussion at http://lambda-the-ultimate.org/
>  > > node/2064
>  > > Which seems to come to a working implementation.
>  > > Usefully, they note that Thompson's original patent has expired, so
>  > > there is no obstacle to using the original, and apparently superior,
>  > > algorithm
>  > >
>  > > I looked at 'man regex' on my Mac, and it describes the POSIX
>  > > implementation of regex. It does contain the note "Originally written
>  > > by Henry Spencer", which is suggested as the basis/culprit of the
>  > > pathological regular expression implementations (I should add, as
>  > > Henry Spencer is one of my heroes, that he is also credited with TCL
>  > > 8 regex, which works very well according to the graphs in the paper).
>  > >
>  > > I have tried grep'ing through *.c in otp_src for regex and regcomp,
>  > > and came up with nothing!
>  > >
>  > > Does anyone know what is used in Erlang? It may be a relatively small
>  > > amount of work to incorporate a new regex to provide more stable
>  > > behaviour.
>  > >
>  > > GB
>  > >
>  > > On 25 Sep 2007, at 10:00,
>  > erlang-questions-request@REDACTED wrote:
>  > >
>  > > > Date: Mon, 24 Sep 2007 09:20:08 -0700 (PDT)
>  > > > From: Thomas Lindgren < thomasl_erlang@REDACTED >
>  > > > Subject: Re: [erlang-questions] Not an Erlang fan
>  > > > To: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
>  > > > Message-ID:
>  > <761781.89097.qm@REDACTED >
>  > > > Content-Type: text/plain; charset=iso-8859-1
>  > > >
>  > > >
>  > > > -
>  > > > Note that this does not include the actual regexp
>  > > > search of each line ... The code to do that could be
>  > > > an FSM + dictionary insert instead of the simple
>  > > > check-and-accumulate for newline. Extra cost? Unclear.
>  > > >
>  > > > One could merge that code into the scanning the binary
>  > > > but this should ideally be done by a regexp compiler,
>  > > > shouldn't it? Not done at this time AFAIK.
>  > > >
>  > > > Best,
>  > > > Thomas
>  > >
>  > > > Date: Mon, 24 Sep 2007 19:55:12 +0200
>  > > > From: Claes Wikstrom < klacke@REDACTED>
>  > > > Subject: Re: [erlang-questions] Not an Erlang fan
>  > > > To: Bob Ippolito < bob@REDACTED>
>  > > > Cc: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
>  > > > Message-ID: < 46F7FA00.8070507@REDACTED>
>  > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>  > > >
>  > > > Bob Ippolito wrote:
>  > > >
>  > > >> On 9/24/07, Patrick Logan < patrickdlogan@REDACTED> wrote:
>  > > >>
>  > > >>>>>>
>  > http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
>  > > >>>>>>
>  > > >>>>>> Tim Bray might raise some valid points here, even if he's
>  > > >>>>>> slightly
>  > > >>>>>> biased by his background.
>  > > >>>>>>
>  > > >
>  > > > No, the only fast way today to process a large file line/by/line is
> to
>  > > >
>  > > > 1. file:open(Filename, [read, raw])
>  > > > 2. In a loop {ok, Bin} = file:read(Fd, BufSize),
>  > > > 3. Use a binary regex matcher such as
>  > > >     http://yaws.hyber.org/download/posregex-1.0.tgz
>  > > >
>  > > > (I don't know the state of the regex lib in OTP today, last time
>  > > >   I looked it sucked bigtime though)
>  > > >
>  > > > /klacke
>  > > >
>  > > _______________________________________________
>  > > erlang-questions mailing list
>  > > erlang-questions@REDACTED
>  > >
> http://www.erlang.org/mailman/listinfo/erlang-questions
>  > >
>  >
>  >
>  > _______________________________________________
>  > erlang-questions mailing list
>  > erlang-questions@REDACTED
>  > http://www.erlang.org/mailman/listinfo/erlang-questions
>  >
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From Tim.Bray@REDACTED  Thu Oct  4 22:55:23 2007
From: Tim.Bray@REDACTED (Tim Bray)
Date: Thu, 04 Oct 2007 13:55:23 -0700
Subject: [erlang-questions] Can't build on new Sun T2?
Message-ID: <9D6DA240-9693-4232-BCC8-C3AF63200494@Sun.COM>

Any advice?  This should be a good erlang box, I want to run some tests.

sca12-3200a-32 ~/dev/otp_src_R11B-5/> uname -a
SunOS sca12-3200a-32 5.10 Generic_120011-13 sun4v sparc SUNW,SPARC- 
Enterprise-****

./configure showed no problems.

sca12-3200a-32 ~/dev/otp_src_R11B-5/> gmake
... hundreds of lines omitted ...

ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
0xfcec0f6e is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfcec0fdb is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfcec0fe1 is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfcec0fe5 is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfcec0fe9 is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
0xfcfbf232 is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfcfbf35d is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
0xfd0012f6 is non-aligned
ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
0xfd001316 is non-aligned
collect2: ld returned 1 exit status
gmake[3]: *** [/home/tb151547/dev/otp_src_R11B-5/bin/sparc-sun- 
solaris2.10/beam.hybrid] Error 1
gmake[3]: Leaving directory `/home/tb151547/dev/otp_src_R11B-5/erts/ 
emulator'
gmake[2]: *** [opt] Error 2
gmake[2]: Leaving directory `/home/tb151547/dev/otp_src_R11B-5/erts/ 
emulator'
gmake[1]: *** [hybrid] Error 2
gmake[1]: Leaving directory `/home/tb151547/dev/otp_src_R11B-5/erts'
gmake: *** [emulator] Error 2

Here's the actual gcc line that blew up:

gcc -o /home/tb151547/dev/otp_src_R11B-5/bin/sparc-sun-solaris2.10/ 
beam.hybrid \
    obj/sparc-sun-solaris2.10/opt/hybrid/erl_main.o              obj/ 
sparc-sun-solaris2.10/opt/hybrid/preload.o obj/sparc-sun-solaris2.10/ 
opt/hybrid/erl_pbifs.o         obj/sparc-sun-solaris2.10/opt/hybrid/ 
benchmark.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_alloc.o               obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_mtrace.o obj/sparc-sun-solaris2.10/opt/hybrid/erl_alloc_util.o  
obj/sparc-sun-solaris2.10/opt/hybrid/erl_goodfit_alloc.o obj/sparc- 
sun-solaris2.10/opt/hybrid/erl_bestfit_alloc.o       obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_afit_alloc.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_instrument.o     obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_init.o obj/sparc-sun-solaris2.10/opt/hybrid/erl_atom_table.o    
obj/sparc-sun-solaris2.10/opt/hybrid/erl_bif_table.o obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_bif_ddll.o        obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_bif_guard.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_bif_info.o        obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_op.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_os.o             obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_lists.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_trace.o       obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_wrap.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_trace.o            obj/sparc-sun-solaris2.10/opt/hybrid/copy.o  
obj/sparc-sun-solaris2.10/opt/hybrid/utils.o                obj/sparc- 
sun-solaris2.10/opt/hybrid/bif.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
io.o                    obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_printf_term.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_debug.o         obj/sparc-sun-solaris2.10/opt/hybrid/erl_md5.o  
obj/sparc-sun-solaris2.10/opt/hybrid/erl_message.o               obj/ 
sparc-sun-solaris2.10/opt/hybrid/erl_process.o obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_process_dict.o      obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_arith.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/time.o            obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_time_sup.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
external.o             obj/sparc-sun-solaris2.10/opt/hybrid/dist.o  
obj/sparc-sun-solaris2.10/opt/hybrid/binary.o               obj/sparc- 
sun-solaris2.10/opt/hybrid/erl_db.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_db_util.o                obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_db_hash.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_db_tree.o           obj/sparc-sun-solaris2.10/opt/hybrid/ 
fix_alloc.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
big.o                     obj/sparc-sun-solaris2.10/opt/hybrid/hash.o  
obj/sparc-sun-solaris2.10/opt/hybrid/index.o                obj/sparc- 
sun-solaris2.10/opt/hybrid/atom.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/module.o               obj/sparc-sun-solaris2.10/opt/hybrid/ 
export.o obj/sparc-sun-solaris2.10/opt/hybrid/register.o            
obj/sparc-sun-solaris2.10/opt/hybrid/break.o obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_async.o           obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_lock_check.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/ggc.o                obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_gc.o obj/sparc-sun-solaris2.10/opt/hybrid/erl_nmgc.o            
obj/sparc-sun-solaris2.10/opt/hybrid/erl_posix_str.o obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_bits.o            obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_math.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_fun.o             obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_port.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_term.o          obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_node_tables.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_monitors.o      obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_process_dump.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_obsolete.o     obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_bif_timer.o obj/sparc-sun-solaris2.10/opt/hybrid/erl_port_task.o  
obj/sparc-sun-solaris2.10/opt/hybrid/beam_emu.o               obj/ 
sparc-sun-solaris2.10/opt/hybrid/beam_opcodes.o obj/sparc-sun- 
solaris2.10/opt/hybrid/beam_load.o            obj/sparc-sun- 
solaris2.10/opt/hybrid/beam_bif_load.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/beam_debug.o          obj/sparc-sun-solaris2.10/opt/hybrid/ 
beam_bp.o obj/sparc-sun-solaris2.10/opt/hybrid/beam_catches.o obj/ 
sparc-sun-solaris2.10/opt/hybrid/sys.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/driver_tab.o obj/sparc-sun-solaris2.10/opt/hybrid/unix_efile.o  
obj/sparc-sun-solaris2.10/opt/hybrid/gzio.o obj/sparc-sun-solaris2.10/ 
opt/hybrid/elib_malloc.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
elib_memmove.o obj/sparc-sun-solaris2.10/opt/hybrid/sys_float.o obj/ 
sparc-sun-solaris2.10/opt/hybrid/sys_time.o obj/sparc-sun-solaris2.10/ 
opt/hybrid/erl_poll.kp.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_check_io.kp.o obj/sparc-sun-solaris2.10/opt/hybrid/erl_poll.nkp.o  
obj/sparc-sun-solaris2.10/opt/hybrid/erl_check_io.nkp.o obj/sparc-sun- 
solaris2.10/opt/hybrid/erl_mseg.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/erl_unix_sys_ddll.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
erl_mtrace_sys_wrap.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
hipe_bif0.o obj/sparc-sun-solaris2.10/opt/hybrid/hipe_bif1.o obj/ 
sparc-sun-solaris2.10/opt/hybrid/hipe_bif2.o obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_debug.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/hipe_gc.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
hipe_mode_switch.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
hipe_native_bif.o obj/sparc-sun-solaris2.10/opt/hybrid/hipe_stack.o  
obj/sparc-sun-solaris2.10/opt/hybrid/hipe_sparc.o obj/sparc-sun- 
solaris2.10/opt/hybrid/hipe_sparc_glue.o obj/sparc-sun-solaris2.10/ 
opt/hybrid/hipe_sparc_bifs.o obj/sparc-sun-solaris2.10/opt/hybrid/ 
hipe_sparc_stack.o obj/sparc-sun-solaris2.10/opt/hybrid/efile_drv.o  
obj/sparc-sun-solaris2.10/opt/hybrid/inet_drv.o obj/sparc-sun- 
solaris2.10/opt/hybrid/zlib_drv.o obj/sparc-sun-solaris2.10/opt/ 
hybrid/ram_file_drv.o obj/sparc-sun-solaris2.10/opt/hybrid/ttsl_drv.o  
-ldl -lm   -lsocket -lnsl -lpthread -lcurses -L../lib/internal/sparc- 
sun-solaris2.10 /home/tb151547/dev/otp_src_R11B-5/erts/obj/sparc-sun- 
solaris2.10/libz.a -lsctp -lethread -lpthread -lerts_internal_r




From toby@REDACTED  Fri Oct  5 01:19:32 2007
From: toby@REDACTED (Toby Thain)
Date: Thu, 4 Oct 2007 20:19:32 -0300
Subject: [erlang-questions] Surprising conclusion in . Parallel
	programming environments: less is more
In-Reply-To: <015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
References: <470349D7.2050108@ericsson.com>
	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
Message-ID: <EC0542A2-485D-4DCA-AFCF-EEF2DA1D6E07@smartgames.ca>


On 4-Oct-07, at 2:41 PM, David Mercer wrote:

> Ok:
>> Oh, and with about 6,000 natural languages
>> on Earth, we should most of us be so paralysed by choice that we  
>> don't
>> say anything!
>
> Good point.  I need to go check out some of those other languages  
> to see if
> they would actually be better than English.  Anyone got any  
> suggestions?

To this native English speaker, Latin languages appeal greatly. I'm  
currently learning Brazilian Portuguese, it is as optimal as any  
human language I've been exposed to: Definitely more regular,  
compact, and euphonic than English.

--Toby

>
> I've heard Chinese is really good, because one character can convey an
> entire word, rather than having to string multiple characters  
> together.
>
> On the other hand, the smaller alphabet of English is kind of like  
> a macro
> capability, since you can form new words out of the basic building  
> blocks.
> Maybe a language with a smaller alphabet is better though.  Greek  
> has only
> 24 letters...
>
> Cheers,
>
> David
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From als@REDACTED  Fri Oct  5 02:34:48 2007
From: als@REDACTED (Anthony Shipman)
Date: Fri, 5 Oct 2007 10:34:48 +1000
Subject: [erlang-questions] Using Lisp to extend a Mac IDE for Erlang
In-Reply-To: <1242f34a0710032026r6e943808wb5ac3273155ebf54@mail.gmail.com>
References: <AB032F81-4F53-4ECD-9E8C-B63AE79C04A6@gmail.com>
	<1242f34a0710032026r6e943808wb5ac3273155ebf54@mail.gmail.com>
Message-ID: <200710051034.49058.als@iinet.net.au>

On Thursday 04 October 2007 13:26, Mark Aufflick wrote:
> Sounds like emacs to me :)
>
> On 10/4/07, Joel Reymont <joelr1@REDACTED> wrote:
> > I'm between projects right now (anyone want to hire me?) so I have
> > time to spare. I'm once again thinking of the Mac IDE for Erlang.
> > More specifically, I'm thinking of the language used to extend such
> > an IDE.
> >
> > I'm wondering if there are any objections to an environment that
> > reuses the Distel code and can be extended in Lisp. Sort of like a
> > TextMate for Erlang with Lisp instead of Ruby.

Why not extend it with javascript. The spider monkey interpreter looks good. 

-- 
Anthony Shipman                    Mamas don't let your babies 
als@REDACTED                   grow up to be outsourced.


From bbmaj7@REDACTED  Fri Oct  5 03:21:11 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Fri, 5 Oct 2007 11:21:11 +1000 (EST)
Subject: [erlang-questions] learning mnesia:select syntax
In-Reply-To: <DFB8E030-7F5F-4D43-A591-FB2EA5584A96@omanetti.fi>
Message-ID: <731507.43485.qm@web52004.mail.re2.yahoo.com>


--- Jani Launonen <launoja@REDACTED> wrote:
> Just try these and try making more compicated examples stepwisely. I  
> hope that helped.

That is exactly the approach that I took.

Example from the mnesia doc:

      MatchHead = #employee{name='$1', sex=male, room_no={'$2', '_'}, _='_'},
      Guard = [{'>=', '$2', 220},{'<', '$2', 230}],
      Result = '$1',
      mnesia:select(employee,[{MatchHead, Guard, [Result]}])

I want to modify this to return {name, sex} of every employee with the same
Guard.

MatchHead is easy change sex='$3'.

Now what do I do with Result if I want to get back multiple fields. I expect to
be able to make it return a tuple for each match but could not figure out how.
Nothing I read indicates that this can or cannot be done.

I ended up giving up and using match_object instead.



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From Bruce@REDACTED  Fri Oct  5 04:53:47 2007
From: Bruce@REDACTED (Bruce Fitzsimons)
Date: Fri, 05 Oct 2007 15:53:47 +1300
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <20071004191041.GA17261@chronos.sin>
References: <20071004191041.GA17261@chronos.sin>
Message-ID: <4705A73B.50008@Fitzsimons.org>

Stephan Maka wrote:
> Hi
>
>
> I'm unsure about the use of gen_tcp:controlling_process when I simply
> use a passive socket. Active sockets are no option here, I have strong
> traffic shaping requirements.
>   
Your code starts working when you change the recv length to 0 (e.g. 
unspecified) and stop expecting only one list elemet e.g. B instead of 
[B]. Your use of controlling_process is completely appropriate (think of 
it as moving the socket spout from one process to another, in active 
mode it would start spewing data at the new process.)

The documentation isn't crystal clear on this point as it says: "The 
|Length| argument is only meaningful when the socket is in |raw| mode 
and denotes the number of bytes to read." rather than something like 
"The |Length| argument should only be non-zero when the socket is in 
|raw| mode and then it denotes the number of bytes to read.".

Your socket is not in raw mode so you'll need to manage excess bytes 
yourself by buffering in a process if you want to process in 
protocol-defined-chunks. I've not used raw mode but it may be appropriate.

Cheers,
Bruce


From jcone@REDACTED  Fri Oct  5 07:44:38 2007
From: jcone@REDACTED (James Cone)
Date: Fri, 05 Oct 2007 18:44:38 +1300
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <mailman.236.1191525065.2732.erlang-questions@erlang.org>
References: <mailman.236.1191525065.2732.erlang-questions@erlang.org>
Message-ID: <4705CF46.8000005@eservglobal.com>

Message-ID: <20071004191041.GA17261@REDACTED>

Hello Stephan, et al,

I don't know how to reply to a digest message; please accept my 
apologies if this reply is not joined on.  I am using Thunderbird, and 
am open to suggestions by email.

I have got Stephan's code to work.  The whole code is below.  I made the 
following gratuitous changes:
   - make the host to connect to into a parameter

I had to make the following changes:
   - add {packet, raw} to the connect options
       - vaguely explained under recv on the gen_tcp page
       - explained better on the inet page

   - use the beginners' syntax for spawn_link
       - I have no idea why this is important, but I couldn't get the 
other one to go

   - complete the loops in doListen and receiver
       - just to get all of the characters

   - remove the io:format from doListen
       - there seems to be locking in the io, so that io from receiver 
can cause starvation in doListen

The code has the following remaining problems:
   - receiver ends disgracefully, when the other end closes the socket 
(trivial)

   - when receiver ends, it takes out doListen, so some of the received 
characters may not be processed

I am confused about Stephan's goal, and whether one byte is an 
appropriate buffer size; it may be for demonstration purposes only.

I am confused about whether there is a better way (documented in inet or 
somewhere) of causing TCP flow-control on the socket.

Regards,
James.

=======================================

-module(tcp_ebadf).

-export([start/1, receiverStart/2]).


start(Host) ->
     io:format("Connecting...~n"),
     {ok, Socket} = gen_tcp:connect(Host, 80,
                                    [{active, false}, {packet, raw}]),

     io:format("Sending...~n"),
     gen_tcp:send(Socket, "GET / HTTP/1.1\r\n"
                  "Host: www.erlang.org\r\n"
                  "Connection: close\r\n"
                  "\r\n"),

     Pid = spawn_link(?MODULE, receiverStart, [self(), Socket]),

%    Pid = spawn_link(fun() ->
%                            receive
%                                go ->
%                                    ok
%                            end,
%                            receiver(self(), Socket)
%                    end),
     ok = gen_tcp:controlling_process(Socket, Pid),
     Pid ! go,

     io:format("Waiting...~n"),
     doListen(Socket).

doListen(Socket) ->
     receive
         {byte, _B} ->
             io:format("Ok, I received a byte: ~p ~n", [[_B]])
     after 10000 ->
             gen_tcp:close(Socket),
             exit(timeout)
     end,
     doListen(Socket).


receiverStart(Listener, Socket) ->
     io:format("Starting with ~w, ~w~n", [Listener, Socket]),
     receive
       go -> ok
     end,
     receiver(Listener, Socket).

receiver(Listener, Socket) ->
%    io:format("Receiving...~n"),
     {ok, [B]} = gen_tcp:recv(Socket, 1),
     Listener ! {byte, B},
     receiver(Listener, Socket).


From per@REDACTED  Fri Oct  5 08:15:46 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 5 Oct 2007 08:15:46 +0200 (CEST)
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <4705A73B.50008@Fitzsimons.org>
Message-ID: <200710050615.l956FkCk013287@pluto.hedeland.org>

Fitzsimons <Bruce@REDACTED> wrote:
>
>Stephan Maka wrote:
>> Hi
>>
>>
>> I'm unsure about the use of gen_tcp:controlling_process when I simply
>> use a passive socket. Active sockets are no option here, I have strong
>> traffic shaping requirements.
>>   
>Your code starts working when you change the recv length to 0 (e.g. 
>unspecified) and stop expecting only one list elemet e.g. B instead of 
>[B].

No it doesn't - but it starts working when self() is evaluated in the
original process instead of in the spawned one, i.e.

@@ -12,12 +12,13 @@
                  "Host: www.erlang.org\r\n"
                  "Connection: close\r\n"
                  "\r\n"),
+    Master = self(),
     Pid = spawn_link(fun() ->
                              receive
                                  go ->
                                      ok
                              end,
-                             receiver(self(), Socket)
+                             receiver(Master, Socket)
                      end),
     ok = gen_tcp:controlling_process(Socket, Pid),
     Pid ! go,

>The documentation isn't crystal clear on this point as it says: "The 
>|Length| argument is only meaningful when the socket is in |raw| mode 
>and denotes the number of bytes to read." rather than something like 
>"The |Length| argument should only be non-zero when the socket is in 
>|raw| mode and then it denotes the number of bytes to read.".
>
>Your socket is not in raw mode so you'll need to manage excess bytes 
>yourself by buffering in a process if you want to process in 
>protocol-defined-chunks. I've not used raw mode but it may be appropriate.

But the socket *is* in raw mode in the sense referred to here, i.e.
{packet, raw}, since that is the ("obvious") default. Though of course
he should be using length 0 in any case, since 1 will be very
inefficient and any other value is unlikely to have the wanted behaviour
in this case (I think the manual is crystal clear on *that* aspect at
least, you just need to read it:-).

--Per



From kostis@REDACTED  Fri Oct  5 08:26:11 2007
From: kostis@REDACTED (Kostis Sagonas)
Date: Fri, 05 Oct 2007 09:26:11 +0300
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <9D6DA240-9693-4232-BCC8-C3AF63200494@Sun.COM>
References: <9D6DA240-9693-4232-BCC8-C3AF63200494@Sun.COM>
Message-ID: <4705D903.8020300@cs.ntua.gr>

Tim Bray wrote:
> Any advice?  This should be a good erlang box, I want to run some tests.
> 
> sca12-3200a-32 ~/dev/otp_src_R11B-5/> uname -a
> SunOS sca12-3200a-32 5.10 Generic_120011-13 sun4v sparc SUNW,SPARC- 
> Enterprise-****
> 
> ./configure showed no problems.
> 
> sca12-3200a-32 ~/dev/otp_src_R11B-5/> gmake
> ... hundreds of lines omitted ...
> 
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
> solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
> 0xfcec0f6e is non-aligned
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
> solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
> 0xfcec0fdb is non-aligned
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
> solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  
> 0xfcec0fe1 is non-aligned
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
> solaris2.10/opt/hybrid/hipe_sparc_bifs.o: symbol <unknown>: offset  


If you want hipe support you (your gcc) needs to use "gas" (the GNU 
assembler), not "as" (the SUN assembler).

You have two options:
   1. Force use of "gas"  (recommended)
   2. Configure with --disable-hipe

Kostis


From bengt.kleberg@REDACTED  Fri Oct  5 08:49:53 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Fri, 05 Oct 2007 08:49:53 +0200
Subject: [erlang-questions] Surprising conclusion in .
 Parallel	programming environments: less is more
In-Reply-To: <015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
References: <470349D7.2050108@ericsson.com>	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
Message-ID: <4705DE91.2080004@ericsson.com>

greetings,

would you mind explaining what you want the new language to be better _at_?
otherwise it is really easy to suggest swedish since it is much better 
at talking to swedes :-)


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-04 19:41, David Mercer wrote:
> Ok:
>> Oh, and with about 6,000 natural languages
>> on Earth, we should most of us be so paralysed by choice that we don't
>> say anything!
> 
> Good point.  I need to go check out some of those other languages to see if
> they would actually be better than English.  Anyone got any suggestions?
> 
> I've heard Chinese is really good, because one character can convey an
> entire word, rather than having to string multiple characters together.
> 
> On the other hand, the smaller alphabet of English is kind of like a macro
> capability, since you can form new words out of the basic building blocks.
> Maybe a language with a smaller alphabet is better though.  Greek has only
> 24 letters...
> 
> Cheers,
> 
> David
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From mikpe@REDACTED  Fri Oct  5 08:56:03 2007
From: mikpe@REDACTED (Mikael Pettersson)
Date: Fri, 5 Oct 2007 08:56:03 +0200 (MEST)
Subject: [erlang-questions] Can't build on new Sun T2?
Message-ID: <200710050656.l956u3QQ022978@harpo.it.uu.se>

On Thu, 04 Oct 2007 13:55:23 -0700, Tim Bray wrote:
> sca12-3200a-32 ~/dev/otp_src_R11B-5/> uname -a
> SunOS sca12-3200a-32 5.10 Generic_120011-13 sun4v sparc SUNW,SPARC- 
> Enterprise-****
> 
> ./configure showed no problems.
> 
> sca12-3200a-32 ~/dev/otp_src_R11B-5/> gmake
> ... hundreds of lines omitted ...
> 
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
> solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
> 0xfcec0f6e is non-aligned
> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
etc

Here we go again. If you look in the mailing list archives you
will find that this issue is reported a few times per year.

It's a toolchain bug. You're using a gcc that's been configured
to use GNU as but Sun's ld. That combination fails to handle some
of the assembly code in the native-code compiler's runtime system,
resulting in the linker errors you quoted.

Rebuilding gcc to use Sun's as and Sun's ld solves the problem.

(Also, the issue doesn't occur on SPARC/Linux where gcc uses
GNU as and GNU ld. It's only GNU as + Sun's ld that bugs out.)

The native code compiler's SPARC backend and runtime system are
scheduled for a complete rewrite later this year, at which time
this issue _may_ finally be solved (or not, since we don't know
why the issue occurs in the first place).

The linker errors can be avoided by doing ./configure --disable-hipe
before make, but then you lose the native-code Erlang compiler.

/Mikael


From huss01@REDACTED  Fri Oct  5 08:56:49 2007
From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=)
Date: Fri, 5 Oct 2007 08:56:49 +0200
Subject: [erlang-questions] Surprising conclusion in . Parallel
	programming environments: less is more
In-Reply-To: <015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
References: <470349D7.2050108@ericsson.com>
	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
Message-ID: <fa53175f0710042356w35d44d3aqf0ca62625f6015c8@mail.gmail.com>

On 10/4/07, David Mercer <dmercer@REDACTED> wrote:
> Ok:
> > Oh, and with about 6,000 natural languages
> > on Earth, we should most of us be so paralysed by choice that we don't
> > say anything!
>
> Good point.  I need to go check out some of those other languages to see if
> they would actually be better than English.  Anyone got any suggestions?
>
Esperanto, Ido, Interlingua, ...

Regards,
/H?kan


From huss01@REDACTED  Fri Oct  5 09:15:15 2007
From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=)
Date: Fri, 5 Oct 2007 09:15:15 +0200
Subject: [erlang-questions] learning mnesia:select syntax
In-Reply-To: <731507.43485.qm@web52004.mail.re2.yahoo.com>
References: <DFB8E030-7F5F-4D43-A591-FB2EA5584A96@omanetti.fi>
	<731507.43485.qm@web52004.mail.re2.yahoo.com>
Message-ID: <fa53175f0710050015y4aa32627o7770afc63c8503c4@mail.gmail.com>

On 10/5/07, Richard Andrews <bbmaj7@REDACTED> wrote:
>
>       MatchHead = #employee{name='$1', sex=male, room_no={'$2', '_'}, _='_'},
>       Guard = [{'>=', '$2', 220},{'<', '$2', 230}],
>       Result = '$1',
>       mnesia:select(employee,[{MatchHead, Guard, [Result]}])
>
> I want to modify this to return {name, sex} of every employee with the same
> Guard.
>
> MatchHead is easy change sex='$3'.
>
> Now what do I do with Result if I want to get back multiple fields.

Result = {{'$1','$3'}},

Perhaps not obvious, but read the match spec information in the ERTS user guide.

Regards,
/H?kan


From jcone@REDACTED  Fri Oct  5 09:22:30 2007
From: jcone@REDACTED (James Cone)
Date: Fri, 05 Oct 2007 20:22:30 +1300
Subject: [erlang-questions] Bad value on output port 'tcp_inet'
In-Reply-To: <470350D5.4090800@hyber.org>
References: <470326ED.6020107@eservglobal.com>
	<b447a30d0710030112p75743a68l22a4c905780d9eb6@mail.gmail.com>
	<470350D5.4090800@hyber.org>
Message-ID: <4705E636.6070502@eservglobal.com>

In my case, it was an integer.  I found out when I captured the error 
and printed the data, so your advice was helpful.

I had a list of strings, and tried to send them one at a time, but the 
last one was a list of integers, so I didn't realise I was fully un-nested.

There's been some discussion in an ejabberd context, of having a 
separate string type.  I think this mistake of mine is another reason 
for having one, and also that I get *lots* of situations where io:format 
guesses wrongly and I get a list of integers that I don't want.

Claes Wikstrom wrote:
> Chandru wrote:
> 
>>
>> This is most probably because you are sending something other than a
>> (deep) list of integers or binaries. Check the data you are trying to
>> send down the socket.
>>
> 
> Typically atoms,
> 
> is in for example:
> 
> gen_tcp:send(Socket, [Bin1, "foo", some_atom, Bin2]).
> 
> /klacke


From raimo+erlang-questions@REDACTED  Fri Oct  5 09:25:18 2007
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Fri, 5 Oct 2007 09:25:18 +0200
Subject: [erlang-questions] : Surprising conclusion in . Parallel
	programming environments: less is more
In-Reply-To: <fa53175f0710042356w35d44d3aqf0ca62625f6015c8@mail.gmail.com>
References: <470349D7.2050108@ericsson.com>
	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>
	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
	<fa53175f0710042356w35d44d3aqf0ca62625f6015c8@mail.gmail.com>
Message-ID: <20071005072518.GB28301@erix.ericsson.se>

On Fri, Oct 05, 2007 at 08:56:49AM +0200, H?kan Huss wrote:
> On 10/4/07, David Mercer <dmercer@REDACTED> wrote:
> > Ok:
> > > Oh, and with about 6,000 natural languages
> > > on Earth, we should most of us be so paralysed by choice that we don't
> > > say anything!
> >
> > Good point.  I need to go check out some of those other languages to see if
> > they would actually be better than English.  Anyone got any suggestions?
> >
> Esperanto, Ido, Interlingua, ...

... Klingon, ...

> 
> Regards,
> /H?kan
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From pacini@REDACTED  Fri Oct  5 09:30:33 2007
From: pacini@REDACTED (Filippo Pacini)
Date: Fri, 05 Oct 2007 09:30:33 +0200
Subject: [erlang-questions] parameterized module returns badfun
In-Reply-To: <779bf2730710041504m2a0714b9w3068920abe99c987@mail.gmail.com>
References: <779bf2730710041504m2a0714b9w3068920abe99c987@mail.gmail.com>
Message-ID: <4705E819.1050109@sgconsulting.it>

Hi,
I had a similar problem with parametrized modules in erlhive.

Ulf pointed me here:
http://www.erlang.org/pipermail/erlang-questions/2007-July/027672.html

There's a patch to apply to shell.erl without it parameterized modules
don't work on 5.5.5.

cheers,
filippo

YC wrote:
> Hi -
> 
> I tried out the parameterized module example in
> http://www.erlang.org/pipermail/erlang-questions/2007-May/026867.html -
> I can compile the example and even create the module instance, but
> evaluating the functions returns badfun instead.  I'm not sure where the
> issues are - any pointers are appreciated.  I'm using 5.5.5 on linux.
> 
> Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe]
> [kernel-poll:false]
> 
> Eshell V5.5.5  (abort with ^G)
> 1> c(pmod).
> {ok,pmod}
> 2> M = pmod:new(3,4).
> {pmod,3,4}
> 3> M:sum().
> 
> =ERROR REPORT==== 2-Oct-2007::10:04:26 ===
> Error in process <0.30.0> with exit value:
> {{badfun,{{pmod,3,4},sum}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> 
> 
> ** exited: {{badfun,{{pmod,3,4},sum}},
>             [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]} **
> 4> M:new_Y(5).
> 
> =ERROR REPORT==== 2-Oct-2007::10:09:13 ===
> Error in process <0.38.0 > with exit value:
> {{badfun,{{pmod,3,4},new_Y}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> 
> ** exited: {{badfun,{{pmod,3,4},new_Y}},
>             [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]} **
> 5>
> 
> Thanks,
> yc
> 



From yinso.chen@REDACTED  Fri Oct  5 09:37:55 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 5 Oct 2007 00:37:55 -0700
Subject: [erlang-questions] parameterized module returns badfun
In-Reply-To: <4705E819.1050109@sgconsulting.it>
References: <779bf2730710041504m2a0714b9w3068920abe99c987@mail.gmail.com>
	<4705E819.1050109@sgconsulting.it>
Message-ID: <779bf2730710050037v72562052wd78a6987103428e9@mail.gmail.com>

Hi -

thanks for the pointer - somehow I didn't find that msg googling - my bad ;)

Thanks,
yc

On 10/5/07, Filippo Pacini <pacini@REDACTED> wrote:
>
> Hi,
> I had a similar problem with parametrized modules in erlhive.
>
> Ulf pointed me here:
> http://www.erlang.org/pipermail/erlang-questions/2007-July/027672.html
>
> There's a patch to apply to shell.erl without it parameterized modules
> don't work on 5.5.5.
>
> cheers,
> filippo
>
> YC wrote:
> > Hi -
> >
> > I tried out the parameterized module example in
> > http://www.erlang.org/pipermail/erlang-questions/2007-May/026867.html -
> > I can compile the example and even create the module instance, but
> > evaluating the functions returns badfun instead.  I'm not sure where the
> > issues are - any pointers are appreciated.  I'm using 5.5.5 on linux.
> >
> > Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe]
> > [kernel-poll:false]
> >
> > Eshell V5.5.5  (abort with ^G)
> > 1> c(pmod).
> > {ok,pmod}
> > 2> M = pmod:new(3,4).
> > {pmod,3,4}
> > 3> M:sum().
> >
> > =ERROR REPORT==== 2-Oct-2007::10:04:26 ===
> > Error in process <0.30.0> with exit value:
> >
> {{badfun,{{pmod,3,4},sum}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> >
> >
> > ** exited: {{badfun,{{pmod,3,4},sum}},
> >             [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> **
> > 4> M:new_Y(5).
> >
> > =ERROR REPORT==== 2-Oct-2007::10:09:13 ===
> > Error in process <0.38.0 > with exit value:
> >
> {{badfun,{{pmod,3,4},new_Y}},[{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> >
> > ** exited: {{badfun,{{pmod,3,4},new_Y}},
> >             [{shell,apply_fun,3},{shell,exprs,6},{shell,eval_loop,3}]}
> **
> > 5>
> >
> > Thanks,
> > yc
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/3ad3cf6c/attachment.htm>

From yinso.chen@REDACTED  Fri Oct  5 10:10:34 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 5 Oct 2007 01:10:34 -0700
Subject: [erlang-questions] posregex error: RE_drv not found
Message-ID: <779bf2730710050110l319ae58dwa99e73c078003ef6@mail.gmail.com>

Hi -

I downloaded posregex and compile on my machine per the suggestion of the
thread
http://www.erlang.org/pipermail/erlang-questions/2007-September/029492.html.
I'm able to make and sudo make install, but when I try to run it in erlang I
encounter the following message:

2> {ok, RE} = posregex:compile(<<"abc.*def">>,[extended]).
sh: line 0: exec: RE_drv: not found

BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution

And not sure how to proceed from here.  I tried both 1.0 & 1.0.1 with the
same results.

Any insights are appreciated, thanks.

yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/153ffb30/attachment.htm>

From christophe.romain@REDACTED  Fri Oct  5 10:15:31 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Fri, 5 Oct 2007 10:15:31 +0200
Subject: [erlang-questions] Erlang release management
In-Reply-To: <f69c840c0710041054v455c1b7cn216f4ecd9ce53abf@mail.gmail.com>
References: <d791b8790709271221s45b5aa58qe230c85e359f7b85@mail.gmail.com>
	<m1ir5v3iy1.fsf@aegir.wgtn.cat-it.co.nz>
	<4B3109BF-2CE8-4768-A996-AC3188F1AF5C@process-one.net>
	<d791b8790709280730h5487328epf3ca89beb283b316@mail.gmail.com>
	<f69c840c0709281710n2e7bdedfkaedaeafd3888d7e4@mail.gmail.com>
	<9D1F5713-B4DE-4866-87A4-1092CA27AFAF@process-one.net>
	<f69c840c0710041054v455c1b7cn216f4ecd9ce53abf@mail.gmail.com>
Message-ID: <BD8F7DCB-8F33-404E-9A74-F52371C76D36@process-one.net>

> [...] so adding a bit of additional stuff shouldn't be a problem.
> What did you have in mind?

I mean, if one can easily plug its software to an otp like module/ 
behavior (or whatever) automatically included by sinan at build  
stage, then it will be trivial for cean to manage live code upgrade  
for all sinan packaged apps.

by now, cean can package almost every contribution available. but  
there is no common way for handling code upgrade. so while  
applications are not developped/packaged using a defined system like  
sinan, it's impossible to handle live code upgrade except calling  
AppMainModule:code_change and let developers implement that function.

moreover, if packages become aware of the otp release management,  
then we'll have more flexibility and ability to handle them.

what do you think about ?


From Bruce@REDACTED  Fri Oct  5 10:39:29 2007
From: Bruce@REDACTED (Bruce Fitzsimons)
Date: Fri, 05 Oct 2007 21:39:29 +1300
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <200710050615.l956FkCk013287@pluto.hedeland.org>
References: <200710050615.l956FkCk013287@pluto.hedeland.org>
Message-ID: <4705F841.8070702@Fitzsimons.org>

Ouch.

Per Hedeland wrote:
> Fitzsimons <Bruce@REDACTED> wrote:
>   
>> Stephan Maka wrote:
>>     
>>> Hi
>>>
>>>
>>> I'm unsure about the use of gen_tcp:controlling_process when I simply
>>> use a passive socket. Active sockets are no option here, I have strong
>>> traffic shaping requirements.
>>>   
>>>       
>> Your code starts working when you change the recv length to 0 (e.g. 
>> unspecified) and stop expecting only one list elemet e.g. B instead of 
>> [B].
>>     
>
> No it doesn't - but it starts working when self() is evaluated in the
> original process instead of in the spawned one, i.e.
>
>   
My only defense is I said "start working", but I don't think I have a 
leg to stand on. ;-(

> But the socket *is* in raw mode in the sense referred to here, i.e.
> {packet, raw}, since that is the ("obvious") default. Though of course
> he should be using length 0 in any case, since 1 will be very
> inefficient and any other value is unlikely to have the wanted behaviour
> in this case (I think the manual is crystal clear on *that* aspect at
> least, you just need to read it:-).
>
>   
Yes you are right. Apologies to the doc authors. I see now that raw 
specifies no particular buffering so length is important, the other 
options provide auto de-packetisation (for want of a better term) as a 
convenience so length is meaningless. Thanks Per.

Sorry for the noise Stephan (and list).

Back to the original question: where is Stephan's occasional ebadf on 
gen_tcp:recv coming from? It is not reproducible it with this code 
(patched or unpatched). Round 2?

Cheers,
Bruce




From yerl@REDACTED  Fri Oct  5 12:26:06 2007
From: yerl@REDACTED (yerl@REDACTED)
Date: Fri,  5 Oct 2007 12:26:06 +0200
Subject: [erlang-questions] posregex error: RE_drv not found
Message-ID: <mnet2.1191579966.22309.yerl@club-internet.fr>

An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/026ca82b/attachment.htm>

From anders.nygren@REDACTED  Fri Oct  5 13:54:46 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 5 Oct 2007 06:54:46 -0500
Subject: [erlang-questions] posregex error: RE_drv not found
In-Reply-To: <779bf2730710050110l319ae58dwa99e73c078003ef6@mail.gmail.com>
References: <779bf2730710050110l319ae58dwa99e73c078003ef6@mail.gmail.com>
Message-ID: <d3520a2f0710050454p9abc57eqf9ad1294c3d75947@mail.gmail.com>

On 10/5/07, YC <yinso.chen@REDACTED> wrote:
> Hi -
>
> I downloaded posregex and compile on my machine per the suggestion of the
> thread
> http://www.erlang.org/pipermail/erlang-questions/2007-September/029492.html
> .   I'm able to make and sudo make install, but when I try to run it in
> erlang I encounter the following message:
>
> 2> {ok, RE} = posregex:compile(<<"abc.*def">>,[extended]).
> sh: line 0: exec: RE_drv: not found
>
>  BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
>        (v)ersion (k)ill (D)b-tables (d)istribution
>
> And not sure how to proceed from here.  I tried both 1.0 & 1.0.1 with the
> same results.
>
> Any insights are appreciated, thanks.
>
> yc
>

I think Your Erlang system is broken. I recently had a similar problem, and
it was solved by recompiling erlang with a different gcc version.
The problem is that with the bad compiler version erlang is not able to
load any drivers.

If You try
crypto:start()
and it also fails, it is probably the same problem as I had.

/Anders


From per@REDACTED  Fri Oct  5 15:50:23 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 5 Oct 2007 15:50:23 +0200 (CEST)
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <4705F841.8070702@Fitzsimons.org>
Message-ID: <200710051350.l95DoNvB022568@pluto.hedeland.org>

Bruce Fitzsimons <Bruce@REDACTED> wrote:
>
>Back to the original question: where is Stephan's occasional ebadf on 
>gen_tcp:recv coming from? It is not reproducible it with this code 
>(patched or unpatched). Round 2?

I carefully avoided to comment on that.:-) From some quick grep's and
tests, it seems it would have to come from the OS interface down in
inet_drv, i.e. an invalid file descriptor gets passed to recv(2) or
somesuch. E.g. trying to gen_tcp:recv/2 from a closed "Erlang socket"
gives a nice {error, closed}. Maybe some race condition if you manage to
squeeze a gen_tcp call in while the socket is "in the process" of
getting closed? I don't think there is any actual problem in there for
correctly written code though.

--Per


From lancecarlson@REDACTED  Fri Oct  5 15:51:22 2007
From: lancecarlson@REDACTED (Lance Carlson)
Date: Fri, 5 Oct 2007 09:51:22 -0400
Subject: [erlang-questions] Installing CEAN to manage Erlang on Ubuntu Fiesty
Message-ID: <49f64a900710050651n3091c721qe7ba5867cb377610@mail.gmail.com>

I tried last night to download and install CEAN on my ubuntu Fiesty
AMD 64 machine and I had some problems. I finally gave up and tried to
install Erlang from source, but again ran into some make errors. I
finally decided to use apt-get to pull down the necessary files but I
was hoping someone could help me get CEAN running on my machine first
as it would be nice to be able to manage Erlang versions from CEAN.
The problem I get when I try to download the archived version of CEAN
AMD 64 developer version, is that everytime I try to run ./start.sh it
tells me that "system linux-x86 is not recognized". Does anyone have
any suggestions?

TIA!
-Lance


From per@REDACTED  Fri Oct  5 16:06:49 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 5 Oct 2007 16:06:49 +0200 (CEST)
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <200710050656.l956u3QQ022978@harpo.it.uu.se>
Message-ID: <200710051406.l95E6nj4022859@pluto.hedeland.org>

Mikael Pettersson <mikpe@REDACTED> wrote:
>
>On Thu, 04 Oct 2007 13:55:23 -0700, Tim Bray wrote:
>> sca12-3200a-32 ~/dev/otp_src_R11B-5/> uname -a
>> SunOS sca12-3200a-32 5.10 Generic_120011-13 sun4v sparc SUNW,SPARC- 
>> Enterprise-****
>> 
>> ./configure showed no problems.
>> 
>> sca12-3200a-32 ~/dev/otp_src_R11B-5/> gmake
>> ... hundreds of lines omitted ...
>> 
>> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
>> solaris2.10/opt/hybrid/hipe_sparc_glue.o: symbol <unknown>: offset  
>> 0xfcec0f6e is non-aligned
>> ld: fatal: relocation error: R_SPARC_32: file obj/sparc-sun- 
>etc
>
>Here we go again.

Indeed. It used to be the case that building Erlang/OTP just required a
'./configure; [g]make' almost anywhere, but it seems to me that HiPE is
frequently the cause of that procedure failing. Of course HiPE is
great:-), but (in general) it's hardly of interest to someone that is
just getting started with Erlang, and I think it's really unfortunate if
that someone gets stuck already at the build stage because of it. Maybe
--disable-hipe should actually be the default everywhere? (Ducking...:-)

--Per


From christophe.romain@REDACTED  Fri Oct  5 16:20:00 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Fri, 5 Oct 2007 16:20:00 +0200
Subject: [erlang-questions] Installing CEAN to manage Erlang on Ubuntu
	Fiesty
In-Reply-To: <49f64a900710050651n3091c721qe7ba5867cb377610@mail.gmail.com>
References: <49f64a900710050651n3091c721qe7ba5867cb377610@mail.gmail.com>
Message-ID: <3DF62427-58E5-4995-93A6-BFC78B25F4A4@process-one.net>

your problem is simple
you downloaded the x86_64 compiled version
but start script fink you are x86, and seeking linux-x86 directory  
fails. normal.

look at start.sh
we use uname -m
if we detect x86_64 we set value to amd64
that means, we are seeking files from linux-amd64 directory.

i guess uname -m does not return x86_64 on your system.
please can you tell we what this command returns ?


From lancecarlson@REDACTED  Fri Oct  5 16:30:42 2007
From: lancecarlson@REDACTED (Lance Carlson)
Date: Fri, 5 Oct 2007 10:30:42 -0400
Subject: [erlang-questions] Erlang forge?
Message-ID: <49f64a900710050730x5aa3558fl778b9393b454f9d9@mail.gmail.com>

I registered the domains erlangforge.org and .com with the mindset of
creating a community based forge site around erlang. Most forge sites
provide a set of tools for each project (SVN, Mailing lists, ftp to a
homepage, etc.). I was wondering what the general consensus on a site
like this would be and what features it should have. I heard that some
people might just want a simple site that cultivates all the projects
into a easy listing and allow the projects to choose the pre-existing
forge-like tool sets. Others may want something more. Thoughts?


From michael.campbell@REDACTED  Fri Oct  5 16:53:23 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Fri, 5 Oct 2007 10:53:23 -0400
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <200710051406.l95E6nj4022859@pluto.hedeland.org>
References: <200710050656.l956u3QQ022978@harpo.it.uu.se>
	<200710051406.l95E6nj4022859@pluto.hedeland.org>
Message-ID: <811f2f1c0710050753q7eedfd68k8fccf83b9376e1c6@mail.gmail.com>

On 10/5/07, Per Hedeland <per@REDACTED> wrote:
> Mikael Pettersson <mikpe@REDACTED> wrote:

> >Here we go again.


Answering a slightly different issue, it is simply the nature of
mailing lists to recycle questions.  Not everyone tries, nor if they
do try, succeeds in finding the answer to their question before
posting.

The meta-<sigh> here is that the "here we go again" theme is every bit
as old as that to which it is referring.  =)


From dmercer@REDACTED  Fri Oct  5 17:33:07 2007
From: dmercer@REDACTED (David Mercer)
Date: Fri, 5 Oct 2007 10:33:07 -0500
Subject: [erlang-questions] Regular expression library (was Not anErlang
	fan)
In-Reply-To: <17244f480710041124s4dff2b8rf35bec8d2e7976e8@mail.gmail.com>
References: <mailman.202.1190710826.2732.erlang-questions@erlang.org>
	<62CB1DD6-55BA-4822-88E3-46E18822C1EA@gmail.com>
	<3dbc6d1c0709261540m636091ecsafcd387283986c73@mail.gmail.com>
	<17244f480709261607w58108f2k7f1282d397f871b2@mail.gmail.com>
	<3dbc6d1c0710031432o438a9f04h226959bc6e1157a@mail.gmail.com>
	<015f01c806ae$19eba950$891ea8c0@SSI.CORP>
	<17244f480710041124s4dff2b8rf35bec8d2e7976e8@mail.gmail.com>
Message-ID: <01a501c80765$07b43100$891ea8c0@SSI.CORP>

Very very cool.  Worth the read.  Thanks.

Cheers,

David

-----Original Message-----
From: yarivvv@REDACTED [mailto:yarivvv@REDACTED] On Behalf Of Yariv Sadan
Sent: Thursday, October 04, 2007 13:24
To: dmercer@REDACTED
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Regular expression library (was Not anErlang
fan)

You can create its abstract forms and then compile them into a .beam
binary and load it into the VM at runtime. ErlyWeb uses this
extensively. Check out smerl.erl (http://code.google.com/p/erlyweb).

Yariv

On 10/4/07, David Mercer <dmercer@REDACTED> wrote:
>
>
>
>
> How do you create a module on the fly?  Wouldn't you have to write it to
> disk and then compile and load it?  Or is there another way?
>
>
>
> Cheers,
>
>
>
> David
>
>
>
>  ________________________________
>
>
> From: erlang-questions-bounces@REDACTED
> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of
> Robert Virding
>  Sent: Wednesday, October 03, 2007 16:32
>  To: Yariv Sadan
>  Cc: erlang-questions@REDACTED; G Bulmer
>  Subject: Re: [erlang-questions] Regular expression library (was Not
> anErlang fan)
>
>
>
>
> That's not too difficult to do at all. The speed you get would depend on
> which regexp family you do it for.
>
>  Robert
>
>
> On 27/09/2007, Yariv Sadan <yarivsadan@REDACTED> wrote:
>
> I was wondering if it would be possible/beneficial to write a regexp
>  compiler that takes a regexp string and compiles it into an Erlang
>  module.
>
>  You would use it as follows:
>
>  regexp2:compile("f*o+o?", myregexp, [native]),
>  true = myregexp:matches("foo").
>
>  This could be a very fast regexp engine.
>
>  What do you think?
>
>  Yariv
>
>  On 9/26/07, Robert Virding <rvirding@REDACTED> wrote:
>  > The regular expression module which is included in the distribution is
a
>  > simple rather na?ve one written in Erlang which uses a back-tracking
>  > algorithm directly on the AST. It has all the problems Russ Cox
mentioned
> in
>  > his paper. I wrote it long ago when no one was interested in parsing
> regexps
>  > in Erlang. :-)
>  >
>  > I am rewriting it at the moment, still in Erlang but using the same
>  > algorithm, or similar to the one, which Russ describes. It is about the
> same
>  > speed as the existing one for simpler regexps but it can handle all the
>  > pathological cases as it should. It will handle POSIX syntax and
> semantics
>  > (but no back-references*). There will also be a version which will
> implement
>  > a subset of PERL regexps with PERL semantics. Both will work on either
> lists
>  > or binaries.
>  >
>  > I hadn't seen the lambda-the-ultimate discussion you mentioned and will
>  > check it out.
>  >
>  > There are a number of regexp packages for Erlang which use PCRE or
other
>  > libraries which I don't have references to. Look in the "Not an Erlang
> fan"
>  > thread here which gives some references to them.
>  >
>  > An interesting alternative would be to write the regexp parser/compiler
> in
>  > Erlang and have the basic regexp matcher in the emulator. This actually
>  > wouldn't be so big, I think. Get the best of both worlds. An I used to
>  > complain (still do in fact) that the emulator is to big. :-)
>  >
>  > Robert
>  >
>  > * Actually there are two types of POSIX regexps in the standard, one
> which
>  > has back-references and one which doesn't.
>  >
>  >
>  >
>  > On 25/09/2007, G Bulmer < gbulmer@REDACTED> wrote:
>  > > After reading the original thread, I wondered what implementation of
>  > > regular expression matching is used by Erlang?
>  > >
>  > > I ask because this paper points out that many implementations have
>  > > bad pathological behaviour:
>  > > http://swtch.com/~rsc/regexp/regexp1.html
>  > > An observation they make is:
>  > > "Regular expression matching can be simple and fast, using finite
>  > > automata-based techniques that have been known for decades. In
>  > > contrast, Perl, PCRE, Python, Ruby, Java, and many other languages
>  > > have regular expression implementations based on recursive
>  > > backtracking that are simple but can be excruciatingly slow."
>  > >
>  > > This is followed by a discussion at http://lambda-the-ultimate.org/
>  > > node/2064
>  > > Which seems to come to a working implementation.
>  > > Usefully, they note that Thompson's original patent has expired, so
>  > > there is no obstacle to using the original, and apparently superior,
>  > > algorithm
>  > >
>  > > I looked at 'man regex' on my Mac, and it describes the POSIX
>  > > implementation of regex. It does contain the note "Originally written
>  > > by Henry Spencer", which is suggested as the basis/culprit of the
>  > > pathological regular expression implementations (I should add, as
>  > > Henry Spencer is one of my heroes, that he is also credited with TCL
>  > > 8 regex, which works very well according to the graphs in the paper).
>  > >
>  > > I have tried grep'ing through *.c in otp_src for regex and regcomp,
>  > > and came up with nothing!
>  > >
>  > > Does anyone know what is used in Erlang? It may be a relatively small
>  > > amount of work to incorporate a new regex to provide more stable
>  > > behaviour.
>  > >
>  > > GB
>  > >
>  > > On 25 Sep 2007, at 10:00,
>  > erlang-questions-request@REDACTED wrote:
>  > >
>  > > > Date: Mon, 24 Sep 2007 09:20:08 -0700 (PDT)
>  > > > From: Thomas Lindgren < thomasl_erlang@REDACTED >
>  > > > Subject: Re: [erlang-questions] Not an Erlang fan
>  > > > To: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
>  > > > Message-ID:
>  > <761781.89097.qm@REDACTED >
>  > > > Content-Type: text/plain; charset=iso-8859-1
>  > > >
>  > > >
>  > > > -
>  > > > Note that this does not include the actual regexp
>  > > > search of each line ... The code to do that could be
>  > > > an FSM + dictionary insert instead of the simple
>  > > > check-and-accumulate for newline. Extra cost? Unclear.
>  > > >
>  > > > One could merge that code into the scanning the binary
>  > > > but this should ideally be done by a regexp compiler,
>  > > > shouldn't it? Not done at this time AFAIK.
>  > > >
>  > > > Best,
>  > > > Thomas
>  > >
>  > > > Date: Mon, 24 Sep 2007 19:55:12 +0200
>  > > > From: Claes Wikstrom < klacke@REDACTED>
>  > > > Subject: Re: [erlang-questions] Not an Erlang fan
>  > > > To: Bob Ippolito < bob@REDACTED>
>  > > > Cc: "Erlang-Questions \(E-mail\)" < erlang-questions@REDACTED>
>  > > > Message-ID: < 46F7FA00.8070507@REDACTED>
>  > > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>  > > >
>  > > > Bob Ippolito wrote:
>  > > >
>  > > >> On 9/24/07, Patrick Logan < patrickdlogan@REDACTED> wrote:
>  > > >>
>  > > >>>>>>
>  > http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
>  > > >>>>>>
>  > > >>>>>> Tim Bray might raise some valid points here, even if he's
>  > > >>>>>> slightly
>  > > >>>>>> biased by his background.
>  > > >>>>>>
>  > > >
>  > > > No, the only fast way today to process a large file line/by/line is
> to
>  > > >
>  > > > 1. file:open(Filename, [read, raw])
>  > > > 2. In a loop {ok, Bin} = file:read(Fd, BufSize),
>  > > > 3. Use a binary regex matcher such as
>  > > >     http://yaws.hyber.org/download/posregex-1.0.tgz
>  > > >
>  > > > (I don't know the state of the regex lib in OTP today, last time
>  > > >   I looked it sucked bigtime though)
>  > > >
>  > > > /klacke
>  > > >
>  > > _______________________________________________
>  > > erlang-questions mailing list
>  > > erlang-questions@REDACTED
>  > >
> http://www.erlang.org/mailman/listinfo/erlang-questions
>  > >
>  >
>  >
>  > _______________________________________________
>  > erlang-questions mailing list
>  > erlang-questions@REDACTED
>  > http://www.erlang.org/mailman/listinfo/erlang-questions
>  >
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



From alexander.lamb@REDACTED  Fri Oct  5 17:46:55 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Fri, 5 Oct 2007 17:46:55 +0200
Subject: [erlang-questions] Beginners question on pattern matching
Message-ID: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>

Hello List,

I am starting on Erlang. I read most of the new Erlang book. However,  
I am lost with something really simple and probably stupid.

I am reading a file, line by line.

Each line is in the form

"a_code" , a_date

The quotes (") are present in the file, as well as the comma. For  
example:

"C51CBA979E4311D67900201842D2EEE81",2003-08-07

I would like to cut the string into pieces (to be able to modify  
it...). Again, this is a test to learn Erlang, I know how to do this  
in two lines of Ruby :-). I could also probably use a built in  
function to parse the string. But I would like to understand pattern  
matching and lists.

So I wrote this, first read the file:

start() ->

	case file:open("ENRL_ENROLLMENT.csv",read) of

		{ok, S} 	->
				Val = do_read(S),
				file:close(S),
				Result = process_lines(Val),
				{ok,Result};

		{error, Why} 	->

				{error, Why}

	end.

Then the lines:

do_read(S) ->

	case io:get_line(S,'') of

		eof		-> [];
		Line 		-> [Line | do_read(S)]

	end.

Then process each line in the list of lines:

process_lines(Lines) ->

	[Date || [_Code | "," ++ Date] <- Lines].


Something is wrong, because it compiles but here is what I get:

11> c(csv_convert.erl).
{ok,csv_convert}
12> csv_convert:start().
{ok,[]}

If each line is actually a list of characters, why can't I match the  
pattern I want???

Thanks for any hint (or examples) !!

Alex
--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/1ae6f33e/attachment.htm>

From erlang@REDACTED  Fri Oct  5 16:46:14 2007
From: erlang@REDACTED (Peter Lund)
Date: Fri, 05 Oct 2007 16:46:14 +0200
Subject: [erlang-questions] gen_tcp custom framing
In-Reply-To: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>
References: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>
Message-ID: <47064E36.708@lundata.se>

Joern skrev:
> Dear erlang-questions,
>
> having to deal with a non-inet supported packet format on TCP / IP
> (STX / ETX limited, fixed size header, 7-bit encoded payload length) I
> am wondering what the best practice might be. Right now I spawn_link
> into a passive recv loop - is it possible to add a custom framing
> behaviour to inet and get to use { active, once }?
Is this UCP or CIMD perhaps? :)

Standard practice or not, I do not know, but some number of times, I
have now implemented myself buffering of incoming IP-segments into a
binary and each time new data comes in I try to find one or more complete
packages in this buffer. If I find it I handle the packages.

In R9, I have also forced to match each packet more than one time since
an initial length value read out first, cannot be used later in the same
binary match statement, but I am guessing that this problem now is history
when using R11.

Peter




From chandrashekhar.mullaparthi@REDACTED  Fri Oct  5 18:07:11 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Fri, 5 Oct 2007 17:07:11 +0100
Subject: [erlang-questions] Beginners question on pattern matching
In-Reply-To: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>
References: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>
Message-ID: <b447a30d0710050907j6b7584bfkfa6de223c4bd0dc7@mail.gmail.com>

On 05/10/2007, Alexander Lamb <alexander.lamb@REDACTED> wrote:
> Hello List,
>
> I am starting on Erlang. I read most of the new Erlang book. However, I am
> lost with something really simple and probably stupid.
>
> I am reading a file, line by line.
>
> Each line is in the form
>
> "a_code" , a_date
>
> The quotes (") are present in the file, as well as the comma. For example:
>
> "C51CBA979E4311D67900201842D2EEE81",2003-08-07
>
>
> I would like to cut the string into pieces (to be able to modify it...).
> Again, this is a test to learn Erlang, I know how to do this in two lines of
> Ruby :-). I could also probably use a built in function to parse the string.
> But I would like to understand pattern matching and lists.
>
> So I wrote this, first read the file:
>
> start() ->
>
>  case file:open("ENRL_ENROLLMENT.csv",read) of
>
>  {ok, S} ->
>  Val = do_read(S),
>  file:close(S),
>  Result = process_lines(Val),
>  {ok,Result};
>
>  {error, Why} ->
>
>  {error, Why}
>
>  end.
>
> Then the lines:
>
> do_read(S) ->
>
>  case io:get_line(S,'') of
>
>  eof -> [];
>  Line -> [Line | do_read(S)]
>
>  end.
>
> Then process each line in the list of lines:
>
> process_lines(Lines) ->
>
>  [Date || [_Code | "," ++ Date] <- Lines].
>
>
> Something is wrong, because it compiles but here is what I get:
>
> 11> c(csv_convert.erl).
> {ok,csv_convert}
> 12> csv_convert:start().
> {ok,[]}
>
> If each line is actually a list of characters, why can't I match the pattern
> I want???
>
> Thanks for any hint (or examples) !!
>

For pattern matching with lists, you have to specify how to match each
element of the list. The way you've written your code, your are
looking for a regular expression capability in your pattern match
which doesn't exist.

Your code can be easily made to work like this.

do_read(S) ->
    case io:get_line(S,'') of
	eof -> [];
	Line -> [string:tokens(Line, ",") | do_read(S)]     %% Change here
    end.

process_lines(Lines) ->
    [Date || [_Code, Date] <- Lines].      %% and a change here.

cheers
Chandru


From christophe.romain@REDACTED  Fri Oct  5 18:15:58 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Fri, 5 Oct 2007 18:15:58 +0200
Subject: [erlang-questions] Installing CEAN to manage Erlang on Ubuntu
	Fiesty
In-Reply-To: <3DF62427-58E5-4995-93A6-BFC78B25F4A4@process-one.net>
References: <49f64a900710050651n3091c721qe7ba5867cb377610@mail.gmail.com>
	<3DF62427-58E5-4995-93A6-BFC78B25F4A4@process-one.net>
Message-ID: <E71D1B1F-E120-4687-A8D5-D25A22C60E48@process-one.net>

For the record,
Lance installed the 32bit version of ubuntu.
using linux x86 CEAN works.


From bob@REDACTED  Fri Oct  5 18:23:55 2007
From: bob@REDACTED (Bob Ippolito)
Date: Fri, 5 Oct 2007 09:23:55 -0700
Subject: [erlang-questions] Beginners question on pattern matching
In-Reply-To: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>
References: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>
Message-ID: <6a36e7290710050923i16f6fae4obf7bda4788fce287@mail.gmail.com>

On 10/5/07, Alexander Lamb <alexander.lamb@REDACTED> wrote:
> Hello List,
>
> I am starting on Erlang. I read most of the new Erlang book. However, I am
> lost with something really simple and probably stupid.
>
> I am reading a file, line by line.
>
> Each line is in the form
>
> "a_code" , a_date
>
> The quotes (") are present in the file, as well as the comma. For example:
>
> "C51CBA979E4311D67900201842D2EEE81",2003-08-07
>
>
> I would like to cut the string into pieces (to be able to modify it...).
> Again, this is a test to learn Erlang, I know how to do this in two lines of
> Ruby :-). I could also probably use a built in function to parse the string.
> But I would like to understand pattern matching and lists.
>
> So I wrote this, first read the file:
>
> start() ->
>
>  case file:open("ENRL_ENROLLMENT.csv",read) of
>
>  {ok, S} ->
>  Val = do_read(S),
>  file:close(S),
>  Result = process_lines(Val),
>  {ok,Result};
>
>  {error, Why} ->
>
>  {error, Why}
>
>  end.
>
> Then the lines:
>
> do_read(S) ->
>
>  case io:get_line(S,'') of
>
>  eof -> [];
>  Line -> [Line | do_read(S)]
>
>  end.

This isn't tail recursion, so your process will probably crash if you
read a very long file. You'll want to rewrite this to use an
accumulator and then lists:reverse. You could even process the lines
as you read them to save you from creating a second big list.

> Then process each line in the list of lines:
>
> process_lines(Lines) ->
>
>  [Date || [_Code | "," ++ Date] <- Lines].
>
>
> Something is wrong, because it compiles but here is what I get:
>
> 11> c(csv_convert.erl).
> {ok,csv_convert}
> 12> csv_convert:start().
> {ok,[]}
>
> If each line is actually a list of characters, why can't I match the pattern
> I want???

Because the pattern you want doesn't mean what you think it does, and
list comprehensions ignore bad matches:

1> [_Code | "," ++ Date] = "Code,Date".
** exited: {{badmatch,"Code,Date"},[{erl_eval,expr,3}]} **
2> [_Code | "," ++ Date] = "C,Date".
"C,Date"
3> {_Code, Date}.
{67,"Date"}

You can't actually write a pattern that matches the tail of a list
with a variable length head. If you knew exactly how long the code was
going to be you could write a very ugly pattern though.

1> Code ++ "," ++ Date = "Code,Date".
** 1: illegal pattern **
2> [_,_,_,_,$, | Date] = "Code,Date".
"Code,Date"
3> Date.
"Date"

You should probably write a function to do it though..

1> tl(lists:dropwhile(fun (C) -> C =/= $, end, "Code,Date")).
"Date"

-bob


From stephan@REDACTED  Fri Oct  5 20:15:52 2007
From: stephan@REDACTED (Stephan Maka)
Date: Fri, 5 Oct 2007 20:15:52 +0200
Subject: [erlang-questions] gen_tcp and processes receiving
In-Reply-To: <200710050615.l956FkCk013287@pluto.hedeland.org>
References: <4705A73B.50008@Fitzsimons.org>
	<200710050615.l956FkCk013287@pluto.hedeland.org>
Message-ID: <20071005181552.GB17261@chronos.sin>

Per and Bruce, thank you for your answers.

Per Hedeland wrote:
> Fitzsimons <Bruce@REDACTED> wrote:
> No it doesn't - but it starts working when self() is evaluated in the
> original process instead of in the spawned one, i.e.
> 
> @@ -12,12 +12,13 @@
>                   "Host: www.erlang.org\r\n"
>                   "Connection: close\r\n"
>                   "\r\n"),
> +    Master = self(),
>      Pid = spawn_link(fun() ->
>                               receive
>                                   go ->
>                                       ok
>                               end,
> -                             receiver(self(), Socket)
> +                             receiver(Master, Socket)
>                       end),
>      ok = gen_tcp:controlling_process(Socket, Pid),
>      Pid ! go,

D'oh, I shouldn't post when I'm in a hurry.

However, I've not been able to reproduce the ebadf fault in this tiny
program.  I will continue research on this. I vaguely suspect SMP
triggering this problem.


Stephan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/03d27db1/attachment.bin>

From joelr1@REDACTED  Fri Oct  5 20:19:58 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Fri, 5 Oct 2007 19:19:58 +0100
Subject: [erlang-questions] Erlang AST with locations
Message-ID: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>

The result of erl_scan:string/1 has line numbers in it but these get  
lost once erl_parse:parse/1 runs.

I would like to have an AST with locations, is there a way to  
accomplish that?

On a related note, I think it's a bummer that there's no  
erl_scan:binary/1.

	Thanks, Joel

--
http://wagerlabs.com







From per@REDACTED  Fri Oct  5 21:27:38 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 5 Oct 2007 21:27:38 +0200 (CEST)
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <811f2f1c0710050753q7eedfd68k8fccf83b9376e1c6@mail.gmail.com>
Message-ID: <200710051927.l95JRcbV029582@pluto.hedeland.org>

"Michael Campbell" <michael.campbell@REDACTED> wrote:
>
>On 10/5/07, Per Hedeland <per@REDACTED> wrote:
>> Mikael Pettersson <mikpe@REDACTED> wrote:
>
>> >Here we go again.
>
>
>Answering a slightly different issue, it is simply the nature of
>mailing lists to recycle questions.  Not everyone tries, nor if they
>do try, succeeds in finding the answer to their question before
>posting.
>
>The meta-<sigh> here is that the "here we go again" theme is every bit
>as old as that to which it is referring.  =)

Good point, and I'll raise you one: If the (non-meta) <sigh>, in
response to a user that is encountering absurdly weird errors(*),
whether he tried to find the answer in the mailing list archives and
failed or simply couldn't bother, comes from one of the people working
on the component that causes these absurdly weird errors to happen in
the first place, it's bordering on being offensive.

OK, to be a bit constructive - wouldn't it be possible to have a
configure test for HiPE that identified these kinds of problems, and
caused HiPE to be disabled with an appropriate warning message, without
failing the build altogether?

--Per Hedeland

(*) I have built and installed gazillions of versions of gazillions of
different open-source packages since long before the term "open source"
was coined, and I'm still baffled at the things the HiPE build
occasionally produces - I really can't recall seeing anything like it
in any other package build, ever.


From yinso.chen@REDACTED  Fri Oct  5 21:43:06 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 5 Oct 2007 12:43:06 -0700
Subject: [erlang-questions] posregex error: RE_drv not found
In-Reply-To: <d3520a2f0710050454p9abc57eqf9ad1294c3d75947@mail.gmail.com>
References: <779bf2730710050110l319ae58dwa99e73c078003ef6@mail.gmail.com>
	<d3520a2f0710050454p9abc57eqf9ad1294c3d75947@mail.gmail.com>
Message-ID: <779bf2730710051243v7f582429vf8c18e9e9a7755cf@mail.gmail.com>

Hi -

Thanks for the suggestion, I can load crypto without issues.

Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe]
[kernel-poll:false]

Eshell V5.5.5  (abort with ^G)
1> crypto:start().
ok
2> crypto:rand_bytes(16).
<<117,32,104,64,51,200,141,119,39,189,60,97,57,38,83,207>>

And posregex shows up in the path when I look @ code:get_path(), and
code:priv_dir(posregex) does point to the right location as well.  Are there
other ways of verifying whether my build is broken?  I'm using fc6 and
gcc-4.1.1-51.

Thanks,
yc

On 10/5/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> On 10/5/07, YC <yinso.chen@REDACTED> wrote:
> > Hi -
> >
> > I downloaded posregex and compile on my machine per the suggestion of
> the
> > thread
> >
> http://www.erlang.org/pipermail/erlang-questions/2007-September/029492.html
> > .   I'm able to make and sudo make install, but when I try to run it in
> > erlang I encounter the following message:
> >
> > 2> {ok, RE} = posregex:compile(<<"abc.*def">>,[extended]).
> > sh: line 0: exec: RE_drv: not found
> >
> >  BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
> >        (v)ersion (k)ill (D)b-tables (d)istribution
> >
> > And not sure how to proceed from here.  I tried both 1.0 & 1.0.1 with
> the
> > same results.
> >
> > Any insights are appreciated, thanks.
> >
> > yc
> >
>
> I think Your Erlang system is broken. I recently had a similar problem,
> and
> it was solved by recompiling erlang with a different gcc version.
> The problem is that with the bad compiler version erlang is not able to
> load any drivers.
>
> If You try
> crypto:start()
> and it also fails, it is probably the same problem as I had.
>
> /Anders
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/a6465b7f/attachment.htm>

From cyberlync@REDACTED  Fri Oct  5 21:50:47 2007
From: cyberlync@REDACTED (Eric Merritt)
Date: Fri, 5 Oct 2007 12:50:47 -0700
Subject: [erlang-questions] Erlang release management
In-Reply-To: <BD8F7DCB-8F33-404E-9A74-F52371C76D36@process-one.net>
References: <d791b8790709271221s45b5aa58qe230c85e359f7b85@mail.gmail.com>
	<m1ir5v3iy1.fsf@aegir.wgtn.cat-it.co.nz>
	<4B3109BF-2CE8-4768-A996-AC3188F1AF5C@process-one.net>
	<d791b8790709280730h5487328epf3ca89beb283b316@mail.gmail.com>
	<f69c840c0709281710n2e7bdedfkaedaeafd3888d7e4@mail.gmail.com>
	<9D1F5713-B4DE-4866-87A4-1092CA27AFAF@process-one.net>
	<f69c840c0710041054v455c1b7cn216f4ecd9ce53abf@mail.gmail.com>
	<BD8F7DCB-8F33-404E-9A74-F52371C76D36@process-one.net>
Message-ID: <f69c840c0710051250x3a8a542dg60463b957ee655e4@mail.gmail.com>

> I mean, if one can easily plug its software to an otp like module/
> behavior (or whatever) automatically included by sinan at build
> stage, then it will be trivial for cean to manage live code upgrade
> for all sinan packaged apps.

If the OTP app/module existed then sinan could do this right now.


> by now, cean can package almost every contribution available. but
> there is no common way for handling code upgrade. so while
> applications are not developped/packaged using a defined system like
> sinan, it's impossible to handle live code upgrade except calling
> AppMainModule:code_change and let developers implement that function.

How would this be different then the current way of doing code
upgrade. In the end the developer is going to have to be aware that a
code upgrade is going on. Thats why we have all the code_change
modules in otp behaviors.


> moreover, if packages become aware of the otp release management,
> then we'll have more flexibility and ability to handle them.
>
> what do you think about ?

I think its a good idea. I am looking forward to elaboration.


From yinso.chen@REDACTED  Fri Oct  5 21:59:11 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 5 Oct 2007 12:59:11 -0700
Subject: [erlang-questions] posregex error: RE_drv not found
In-Reply-To: <mnet2.1191579966.22309.yerl@club-internet.fr>
References: <mnet2.1191579966.22309.yerl@club-internet.fr>
Message-ID: <779bf2730710051259v504464d1qc9de984ca0301a85@mail.gmail.com>

That does it!!  Thanks!  I didn't realize I have to load the driver
manually.

It seem to make sense if the loading process can be automated.  Read through
the erl_ddll doc it seems the try_load function can be used even when the
driver is already loaded.  But I'm wondering about the path - it seems that
it actually need a physical path.  I thought erlang already know where the
lib exists, or do I have to manually concat the path and pass to the load
function?

Thanks,
yc

On 10/5/07, yerl@REDACTED <yerl@REDACTED> wrote:
>
>  Hi !
>
> The README isn't clear enough.
> Follow this one:
>
> 1. Compile your regexp.
>
> $ make
> $ erl +Ktrue -pa ebin/
>
> 1> erl_ddll:load_driver("./priv", 'RE_drv').
> 2> erl_ddll:loaded_drivers().
>
> 4>  {ok, RE} = posregex:compile(<<"abc.*foo">>, [extended]).
> {ok,#Port<0.101>}
>
>
> Try to match something
>
> 7> posregex:match(RE, <<"abc mre text here foo">>, []).
> ok
>
>
> If it doesn't match
>
> 9> posregex:match(RE, <<"abdc mre text here foo">>, []).
> {error,nomatch}
>
>
> Try to match and find out where the match occured
>
> 10> posregex:exec(RE, <<"abc mre text here foo">>, []).
> {ok,[{0,21}]}
>
>
> Free memory occupied by the compilation (or exit process since
> RE is an erlang port)
>
> 11> posregex:free(RE).
> ok
>
>
> cheers
> Y.
> ----Message d'origine----
> >Date: Fri, 5 Oct 2007 01:10:34 -0700
> >De: YC
> >A: erlang-questions@REDACTED
> >Sujet: [erlang-questions] posregex error: RE_drv not found
> >
> >
> >
> >Hi -
> >
> >I downloaded posregex and compile on my machine per the suggestion of the
>
> >thread
> >
> http://www.erlang.org/pipermail/erlang-questions/2007-September/029492.html.
>
> >I'm able to make and sudo make install, but when I try to run it in
> erlang I
> >encounter the following message:
> >
> >2> {ok, RE} = posregex:compile(<<"abc.*def">>,[extended]).
> >sh: line 0: exec: RE_drv: not found
> >
> >BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
> > (v)ersion (k)ill (D)b-tables (d)istribution
> >
> >And not sure how to proceed from here. I tried both 1.0 & 1.0.1 with the
> >same results.
> >
> >Any insights are appreciated, thanks.
> >
> >yc
> >
> >_______________________________________________
> >erlang-questions mailing list
> >erlang-questions@REDACTED
> >http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/9c2397da/attachment.htm>

From machinshin2002@REDACTED  Fri Oct  5 21:23:31 2007
From: machinshin2002@REDACTED (Vat Raghavan)
Date: Fri, 5 Oct 2007 12:23:31 -0700 (PDT)
Subject: [erlang-questions] Erlounge in Atlanta, GA
Message-ID: <544064.83898.qm@web31505.mail.mud.yahoo.com>

hey all,

I live in Atlanta GA. 
I'm wondering if there's anyone here, or in the nearby area who'd like to get together for a Erlounge :) 
There are already two people in irc://irc.freenode.net#erlang who's stated their interest. 
Anyone interested & available please either contact me personally, or come into #erlang and mention your interest, my nick in there is 'MachinShin'.





      ____________________________________________________________________________________
Shape Yahoo! in your own image.  Join our Network Research Panel today!   http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 




From vladdu55@REDACTED  Fri Oct  5 22:17:01 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Fri, 5 Oct 2007 20:17:01 +0000
Subject: [erlang-questions] Erlang AST with locations
In-Reply-To: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
References: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
Message-ID: <95be1d3b0710051317l5e818df7ic8aa6d23a6723e86@mail.gmail.com>

Hi,

On 10/5/07, Joel Reymont <joelr1@REDACTED> wrote:
> The result of erl_scan:string/1 has line numbers in it but these get
> lost once erl_parse:parse/1 runs.
>
> I would like to have an AST with locations, is there a way to
> accomplish that?

If the license fits with your purpose, you could look at erlide_scan
and erlide_parse -- in the erlide code, of course. They are modified
versions of the respective OTP modules, that return tokens in which
the position is {{line number, offset in file}, token length}. The
textual form of the token is also returned so that there is a
difference between scanning "15" and "16#F".

best regards,
Vlad


From yinso.chen@REDACTED  Fri Oct  5 22:07:36 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 5 Oct 2007 13:07:36 -0700
Subject: [erlang-questions] posregex error: RE_drv not found
In-Reply-To: <779bf2730710051259v504464d1qc9de984ca0301a85@mail.gmail.com>
References: <mnet2.1191579966.22309.yerl@club-internet.fr>
	<779bf2730710051259v504464d1qc9de984ca0301a85@mail.gmail.com>
Message-ID: <779bf2730710051307h17c2b791kf301c64c8e704eff@mail.gmail.com>

Ah - I figured out that I just need to use
erl_ddll:load_driver(code:priv_dir(posregex),
'RE_drv') to make it work - thanks!
yc

On 10/5/07, YC <yinso.chen@REDACTED> wrote:
>
> That does it!!  Thanks!  I didn't realize I have to load the driver
> manually.
>
> It seem to make sense if the loading process can be automated.  Read
> through the erl_ddll doc it seems the try_load function can be used even
> when the driver is already loaded.  But I'm wondering about the path - it
> seems that it actually need a physical path.  I thought erlang already know
> where the lib exists, or do I have to manually concat the path and pass to
> the load function?
>
> Thanks,
> yc
>
> On 10/5/07, yerl@REDACTED <yerl@REDACTED > wrote:
> >
> >  Hi !
> >
> > The README isn't clear enough.
> > Follow this one:
> >
> > 1. Compile your regexp.
> >
> > $ make
> > $ erl +Ktrue -pa ebin/
> >
> > 1> erl_ddll:load_driver("./priv", 'RE_drv').
> > 2> erl_ddll:loaded_drivers().
> >
> > 4>  {ok, RE} = posregex:compile(<<"abc.*foo">>, [extended]).
> > {ok,#Port<0.101>}
> >
> >
> > Try to match something
> >
> > 7> posregex:match(RE, <<"abc mre text here foo">>, []).
> > ok
> >
> >
> > If it doesn't match
> >
> > 9> posregex:match(RE, <<"abdc mre text here foo">>, []).
> > {error,nomatch}
> >
> >
> > Try to match and find out where the match occured
> >
> > 10> posregex:exec(RE, <<"abc mre text here foo">>, []).
> > {ok,[{0,21}]}
> >
> >
> > Free memory occupied by the compilation (or exit process since
> > RE is an erlang port)
> >
> > 11> posregex:free(RE).
> > ok
> >
> >
> > cheers
> > Y.
> > ----Message d'origine----
> > >Date: Fri, 5 Oct 2007 01:10:34 -0700
> > >De: YC
> > >A: erlang-questions@REDACTED
> > >Sujet: [erlang-questions] posregex error: RE_drv not found
> > >
> > >
> > >
> > >Hi -
> > >
> > >I downloaded posregex and compile on my machine per the suggestion of
> > the
> > >thread
> > >http://www.erlang.org/pipermail/erlang-questions/2007-September/029492.html
> > .
> > >I'm able to make and sudo make install, but when I try to run it in
> > erlang I
> > >encounter the following message:
> > >
> > >2> {ok, RE} = posregex:compile(<<"abc.*def">>,[extended]).
> > >sh: line 0: exec: RE_drv: not found
> > >
> > >BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
> > > (v)ersion (k)ill (D)b-tables (d)istribution
> > >
> > >And not sure how to proceed from here. I tried both 1.0 & 1.0.1 with
> > the
> > >same results.
> > >
> > >Any insights are appreciated, thanks.
> > >
> > >yc
> > >
> > >_______________________________________________
> > >erlang-questions mailing list
> > >erlang-questions@REDACTED
> > >http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071005/374bb40e/attachment.htm>

From rsaccon@REDACTED  Fri Oct  5 22:49:34 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Fri, 5 Oct 2007 17:49:34 -0300
Subject: [erlang-questions] Erlang forge?
In-Reply-To: <49f64a900710050730x5aa3558fl778b9393b454f9d9@mail.gmail.com>
References: <49f64a900710050730x5aa3558fl778b9393b454f9d9@mail.gmail.com>
Message-ID: <eb4177b00710051349v179dc7cbm3e8d62486d890bbe@mail.gmail.com>

Just an idea for quickly creating something useful: do a mashup maybe
easily done with yahoo pipes or google mashup editor.

let me give you an example what you could automate:

this manually performed query
http://code.google.com/hosting/search?q=erlang&btn=Search+Projects
lists now 85 erlang tagged projects.

On September  04 I did that query as well
(http://www.rsaccon.com/2007/09/another-erlang-imaging-library-binding.html)
 and I got 72 projects.

So there is some quickly moving and well structured data (at least in
case of googlecode) which I think is worth observing and publishing on
a easy to remember website (as  the domains you registered)



On 10/5/07, Lance Carlson <lancecarlson@REDACTED> wrote:
> I registered the domains erlangforge.org and .com with the mindset of
> creating a community based forge site around erlang. Most forge sites
> provide a set of tools for each project (SVN, Mailing lists, ftp to a
> homepage, etc.). I was wondering what the general consensus on a site
> like this would be and what features it should have. I heard that some
> people might just want a simple site that cultivates all the projects
> into a easy listing and allow the projects to choose the pre-existing
> forge-like tool sets. Others may want something more. Thoughts?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Roberto Saccon
http://rsaccon.com


From per@REDACTED  Fri Oct  5 22:56:05 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 5 Oct 2007 22:56:05 +0200 (CEST)
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <06DCCDDF-598A-45A5-82FC-8C296706BF22@sun.com>
Message-ID: <200710052056.l95Ku5Oc031564@pluto.hedeland.org>

Tim Bray <Tim.Bray@REDACTED> wrote:
>
>By the way, I went and tried the build again on a different sparc  
>running an (I think) more recent OpenSolaris, and in the erlang build  
>output I saw a lot of "Solaris 2.11" instead of "Solaris 2.10" (!?!)  

Yeah, "11" is the version number Sun has assigned to at least recent
versions of "OpenSolaris" (shouldn't you know that?:-) -

sun> uname -sripv
SunOS 5.11 snv_55b i386 i86pc

- though I guess since 2.7 or thereabouts, the "2." should be dropped
from the "Solaris version" - i.e. it's "Solaris 11".

>and anyhow, it built and runs fine.  So the problem may just go away  
>in the fullness of time.  -Tim

As Mikael wrote, it's a matter of how your gcc toolchain was built -
OpenSolaris comes with a lot of pre-built GNU and other 3rd-party SW,
and apparently it's done right from the POV of HiPE:

sun> gcc -v
Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/specs
Configured with: /builds1/sfwnv-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++,f77 --enable-shared
Thread model: posix
gcc version 3.4.3 (csl-sol210-3_4-20050802)

- whereas if you built and installed gcc yourself, I assume you could
end up either way, i.e. using the Sun assembler or the GNU one, and this
problem may not go away.

--Per



From Tim.Bray@REDACTED  Fri Oct  5 21:40:44 2007
From: Tim.Bray@REDACTED (Tim Bray)
Date: Fri, 05 Oct 2007 12:40:44 -0700
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <200710051927.l95JRcbV029582@pluto.hedeland.org>
References: <200710051927.l95JRcbV029582@pluto.hedeland.org>
Message-ID: <06DCCDDF-598A-45A5-82FC-8C296706BF22@sun.com>

By the way, I went and tried the build again on a different sparc  
running an (I think) more recent OpenSolaris, and in the erlang build  
output I saw a lot of "Solaris 2.11" instead of "Solaris 2.10" (!?!)  
and anyhow, it built and runs fine.  So the problem may just go away  
in the fullness of time.  -Tim

On Oct 5, 2007, at 12:27 PM, Per Hedeland wrote:

> "Michael Campbell" <michael.campbell@REDACTED> wrote:
>>
>> On 10/5/07, Per Hedeland <per@REDACTED> wrote:
>>> Mikael Pettersson <mikpe@REDACTED> wrote:
>>
>>>> Here we go again.
>>
>>
>> Answering a slightly different issue, it is simply the nature of
>> mailing lists to recycle questions.  Not everyone tries, nor if they
>> do try, succeeds in finding the answer to their question before
>> posting.
>>
>> The meta-<sigh> here is that the "here we go again" theme is every  
>> bit
>> as old as that to which it is referring.  =)
>
> Good point, and I'll raise you one: If the (non-meta) <sigh>, in
> response to a user that is encountering absurdly weird errors(*),
> whether he tried to find the answer in the mailing list archives and
> failed or simply couldn't bother, comes from one of the people working
> on the component that causes these absurdly weird errors to happen in
> the first place, it's bordering on being offensive.
>
> OK, to be a bit constructive - wouldn't it be possible to have a
> configure test for HiPE that identified these kinds of problems, and
> caused HiPE to be disabled with an appropriate warning message,  
> without
> failing the build altogether?
>
> --Per Hedeland
>
> (*) I have built and installed gazillions of versions of gazillions of
> different open-source packages since long before the term "open  
> source"
> was coined, and I'm still baffled at the things the HiPE build
> occasionally produces - I really can't recall seeing anything like it
> in any other package build, ever.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From michael.campbell@REDACTED  Fri Oct  5 23:52:56 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Fri, 5 Oct 2007 17:52:56 -0400
Subject: [erlang-questions] Erlounge in Atlanta, GA
In-Reply-To: <544064.83898.qm@web31505.mail.mud.yahoo.com>
References: <544064.83898.qm@web31505.mail.mud.yahoo.com>
Message-ID: <811f2f1c0710051452yabe7e35lc0a7766694d70eb0@mail.gmail.com>

On 10/5/07, Vat Raghavan <machinshin2002@REDACTED> wrote:
> hey all,
>
> I live in Atlanta GA.
> I'm wondering if there's anyone here, or in the nearby area who'd like to get together for a Erlounge :)
> There are already two people in irc://irc.freenode.net#erlang who's stated their interest.
> Anyone interested & available please either contact me personally, or come into #erlang and mention your interest, my nick in there is 'MachinShin'.


I'm one of the 2 ("MoeD"), but as I mentioned in the channel, I'll
show up if timing permits (I travel as part of my employment).


From 0x6e6562@REDACTED  Sat Oct  6 00:24:35 2007
From: 0x6e6562@REDACTED (Ben Hood)
Date: Fri, 5 Oct 2007 23:24:35 +0100
Subject: [erlang-questions] Dynamically dispatching encoded messages to
	functions
In-Reply-To: <269388e30710051524r7baaca8aw6499f8051ce687de@mail.gmail.com>
References: <269388e30710041407s68e49cc6k768655ec7631b3d0@mail.gmail.com>
	<1191570364.13529.35.camel@humm>
	<269388e30710051524r7baaca8aw6499f8051ce687de@mail.gmail.com>
Message-ID: <269388e30710051524v5bfe01bfh5f2b4ad16dcbf503@mail.gmail.com>

> Is what you are doing related in any way to RabbitMQ
> (http://www.rabbitmq.com/), which is already written in Erlang, but does
> not currently have an Erlang client?

Yes. I've implemented an Erlang client for AMQP, but I would like to
add some higher level message passing paradigms such as RPC that would
build on top of the basic AMQP client API. The motivation is to
implement the boilerplate code for these standard interactions in a
generic fashion so that business code has as little AMQP plumbing as
possible. The client is in a beta phase at the moment.

BTW I've started a wiki page on the client
(https://dev.rabbitmq.com/wiki/ErlangClient), but please forgive the
Erlang formatting as I am still getting used to the markup.

Ben


From gbulmer@REDACTED  Sat Oct  6 01:02:40 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sat, 6 Oct 2007 01:02:40 +0200
Subject: [erlang-questions] Surprising conclusion in Parallel
	programming environments: less is more
Message-ID: <386C26B7-3443-4258-957D-4E7D8AC37625@gmail.com>

> Date: Thu, 4 Oct 2007 10:43:12 +0200
> From: "David Welton" <davidnwelton@REDACTED>
> ...
> The Paradox of Choice book is pretty good, and while the above
> conclusions are off, the concept is valuable.
>
> Think of how anxious some people get trying to pick the "best"
> programming language, especially because as someone with less
> experience, they have less skills to even judge.

> Date: Thu, 4 Oct 2007 12:41:09 -0500
> From: "David Mercer"
>
> Ok:
>
>> Oh, and with about 6,000 natural languages
>> on Earth, we should most of us be so paralysed by choice that we  
>> don't
>> say anything!
>>
>
> Good point.  I need to go check out some of those other languages  
> to see if
> they would actually be better than English.  Anyone got any  
> suggestions?
>
> I've heard Chinese is really good, because one character can convey an
> entire word, rather than having to string multiple characters  
> together.
>
> On the other hand, the smaller alphabet of English is kind of like  
> a macro
> capability, since you can form new words out of the basic building  
> blocks.
> Maybe a language with a smaller alphabet is better though.  Greek  
> has only
> 24 letters...
>

'Surely' natural language recognition has developed to the point  
where it can process Esperanto:
http://www.esperanto.net/
so why do we even need the artificial division between natural  
languages and programming languages !-)
Then there *is* a single default for everything; no matter what you  
want to say, you can be similarly well understood everywhere ;-)

GB

PS - Thanks to all of the wonderful folks at the Erlang and CUFP  
workshops. It was a great experience. Night, night, and safe journeys  
home.



From mikpe@REDACTED  Sat Oct  6 01:51:39 2007
From: mikpe@REDACTED (Mikael Pettersson)
Date: Sat, 6 Oct 2007 01:51:39 +0200 (MEST)
Subject: [erlang-questions] Can't build on new Sun T2?
Message-ID: <200710052351.l95NpdHD001561@harpo.it.uu.se>

On Fri, 5 Oct 2007 21:27:38 +0200 (CEST), Per Hedeland wrote:
> Good point, and I'll raise you one: If the (non-meta) <sigh>, in
> response to a user that is encountering absurdly weird errors(*),
> whether he tried to find the answer in the mailing list archives and
> failed or simply couldn't bother, comes from one of the people working
> on the component that causes these absurdly weird errors to happen in
> the first place, it's bordering on being offensive.

There's a bug in Sun's toolchain. Should I as I HiPE developer feel
guilty about triggering it? H*ll no. I did not _cause_ the problem,
Sun did. It's not my job to bend over backwards to accomodate every
**** bug out there, especially when there are viable workaorunds.

Did per@REDACTED ever contribute code to solve this problem? No.

> OK, to be a bit constructive - wouldn't it be possible to have a
> configure test for HiPE that identified these kinds of problems, and
> caused HiPE to be disabled with an appropriate warning message, without
> failing the build altogether?

Given the amount of grief Sun's crap toolchain is causing us,
I wouldn't be opposed to not autoenabling HiPE on Solaris/SPARC.

/Mikael

(highly restrained, could have used a _lot_ more four letter words,
only representing myself not the HiPE group in general)


From jeffm@REDACTED  Sat Oct  6 03:34:13 2007
From: jeffm@REDACTED (jm)
Date: Sat, 06 Oct 2007 11:34:13 +1000
Subject: [erlang-questions] Surprising conclusion in .
 Parallel	programming environments: less is more
In-Reply-To: <4705DE91.2080004@ericsson.com>
References: <470349D7.2050108@ericsson.com>	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>
	<4705DE91.2080004@ericsson.com>
Message-ID: <4706E615.1000504@ghostgun.com>

Heading off topic here....

Bengt Kleberg wrote:
> would you mind explaining what you want the new language to be better _at_?
> otherwise it is really easy to suggest swedish since it is much better 
> at talking to swedes :-)

As a native Swedish speaker you most likely haven't noticed that there 
is a lack of (free) introductory Swedish material on the Internet. This 
seems rather surprising given how up the swedes are on the Internet. 
There's a few Italian, Spanish, Mandarin, German, Greek, even Russian 
and Pinoy (Tagalog) podcasts for foreign language students. For the more 
popular languages 
(http://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers) 
there are even free on-line courses. Search as I might there doesn't 
appear to be much out there for Swedish. Here I would have thought that 
the Swedish govt would have had something as part of promoting "the 
Swedish perspective" much like the BBCs brief from the British govt. 
Strangely enough the BBC actually has quite a good selection of foreign 
language material (http://www.bbc.co.uk/languages/ ). Especially strange 
given that the English sometimes have a reputation for being xenophobic. 
The best I've come across has been fsi 
(http://fsi-language-courses.com/) which looks promising, but really 
don't have any idea as to how good it is in practice having not tried it 
nor knowing anyone who has.

So native Swedish speakers here's you chance to promote the learning of 
Swedish. Speak up and recommend good free Swedish learn material 
available online.


Now I just need find the time.

Jeff.
ps. Just took a quick look at Australia's national broadcaster, the ABC 
(http://www.abc.net.au/), to see how well Australia stacks up with 
English as a second language education on-line. I embarrassed to say not 
very well. I'm hoping it's just that I'm looking it the wrong place. 
Searching the government domains, use site:gov.au on google, does turn 
up lots of bits and pieces mostly by the state govts (State govts are 
responsible for education in Australia). Not sure how much of this is free.



From bbmaj7@REDACTED  Sat Oct  6 03:36:04 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Sat, 6 Oct 2007 11:36:04 +1000 (EST)
Subject: [erlang-questions] learning mnesia:select syntax
In-Reply-To: <fa53175f0710050015y4aa32627o7770afc63c8503c4@mail.gmail.com>
Message-ID: <554873.74860.qm@web52002.mail.re2.yahoo.com>


--- H?kan Huss <huss01@REDACTED> wrote:

> On 10/5/07, Richard Andrews <bbmaj7@REDACTED> wrote:
> >
> >       MatchHead = #employee{name='$1', sex=male, room_no={'$2', '_'},
> _='_'},
> >       Guard = [{'>=', '$2', 220},{'<', '$2', 230}],
> >       Result = '$1',
> >       mnesia:select(employee,[{MatchHead, Guard, [Result]}])
> >
> > I want to modify this to return {name, sex} of every employee with the same
> > Guard.
> >
> > MatchHead is easy change sex='$3'.
> >
> > Now what do I do with Result if I want to get back multiple fields.
> 
> Result = {{'$1','$3'}},
> 
> Perhaps not obvious, but read the match spec information in the ERTS user
> guide.

I did read that information in ERTS and came to the conclusion that it should
be 

Result = {'$1', '$3'}

which didn't work. And the error message of course gave me no information about
how it was malformed.

I cannot tell you how many times I read the ERTS grammar and still didn't
notice the double braces in TermConstruct.

Thanks.



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From bjorn@REDACTED  Sat Oct  6 10:16:30 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 06 Oct 2007 10:16:30 +0200
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <200710052351.l95NpdHD001561@harpo.it.uu.se>
References: <200710052351.l95NpdHD001561@harpo.it.uu.se>
Message-ID: <o1abqwejz5.fsf@erix.ericsson.se>

Mikael Pettersson <mikpe@REDACTED> writes:

> Given the amount of grief Sun's crap toolchain is causing us,
> I wouldn't be opposed to not autoenabling HiPE on Solaris/SPARC.

Good idea. Let's turn off auto-enabling on Solaris/Sparc in R12B.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From per@REDACTED  Sat Oct  6 12:50:57 2007
From: per@REDACTED (Per Hedeland)
Date: Sat, 6 Oct 2007 12:50:57 +0200 (CEST)
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <200710052351.l95NpdHD001561@harpo.it.uu.se>
Message-ID: <200710061050.l96Aovge049061@pluto.hedeland.org>

Mikael Pettersson <mikpe@REDACTED> wrote:
>
>On Fri, 5 Oct 2007 21:27:38 +0200 (CEST), Per Hedeland wrote:
>> Good point, and I'll raise you one: If the (non-meta) <sigh>, in
>> response to a user that is encountering absurdly weird errors(*),
>> whether he tried to find the answer in the mailing list archives and
>> failed or simply couldn't bother, comes from one of the people working
>> on the component that causes these absurdly weird errors to happen in
>> the first place, it's bordering on being offensive.
>
>There's a bug in Sun's toolchain. Should I as I HiPE developer feel
>guilty about triggering it? H*ll no. I did not _cause_ the problem,
>Sun did. It's not my job to bend over backwards to accomodate every
>**** bug out there, especially when there are viable workaorunds.

I didn't suggest that you should - the problem I referred to was not
that HiPE failed to build, but that the requirements of HiPE, which are
clearly stronger than those of the rest of Erlang/OTP or most other
open-source packages (nothing *wrong* with that, of course), caused the
complete Erlang/OTP build to fail.

And while I guess that you were just venting your frustration over the
Sun problem, for me (and apparently I wasn't alone) your message came
across as reproaching the presumably innocent user that came across it.

>Did per@REDACTED ever contribute code to solve this problem? No.

Well, I occasionally contribute code to various open-source packages,
but I have to admit that I normally need the motivation of a problem
that I'm affected by to do so.

>> OK, to be a bit constructive - wouldn't it be possible to have a
>> configure test for HiPE that identified these kinds of problems, and
>> caused HiPE to be disabled with an appropriate warning message, without
>> failing the build altogether?
>
>Given the amount of grief Sun's crap toolchain is causing us,
>I wouldn't be opposed to not autoenabling HiPE on Solaris/SPARC.

Seems like a good idea to me.

--Per


From Tim.Bray@REDACTED  Sat Oct  6 18:02:49 2007
From: Tim.Bray@REDACTED (Tim Bray)
Date: Sat, 06 Oct 2007 09:02:49 -0700
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <o1abqwejz5.fsf@erix.ericsson.se>
References: <200710052351.l95NpdHD001561@harpo.it.uu.se>
	<o1abqwejz5.fsf@erix.ericsson.se>
Message-ID: <C75B073A-F3F9-4E12-9CBD-BFF7FED9FA87@sun.com>


On Oct 6, 2007, at 1:16 AM, Bjorn Gustavsson wrote:

> Mikael Pettersson <mikpe@REDACTED> writes:
>
>> Given the amount of grief Sun's crap toolchain is causing us,
>> I wouldn't be opposed to not autoenabling HiPE on Solaris/SPARC.
>
> Good idea. Let's turn off auto-enabling on Solaris/Sparc in R12B.

Well, it seems to work if version > 10, could it be made  
conditional?  -T




From igouy2@REDACTED  Sat Oct  6 17:44:53 2007
From: igouy2@REDACTED (Isaac Gouy)
Date: Sat, 6 Oct 2007 08:44:53 -0700 (PDT)
Subject: [erlang-questions] garbage collection questions
Message-ID: <614307.45023.qm@web60525.mail.yahoo.com>

James Hague wrote

> Originally the sieve benchmark used lists--which results in some
> pretty Erlang code--but there wasn't consistency between languages.
> Some functional languages used lists for the sieve benchmark, most
> other entries used destructive array updates.  I believe the point of
> nsieve was to require destructive array updates.

Note: nsieve is not quite the same as sieve, and afaict no one has
contributed an Erlang nsieve program that used lists.

Back in the day Doug Bagley wrote a sieve program based on some code
from Maurice Castro's book, and that was shown back in 2004.

nsieve does not /require/ destructive array updates but in many
languages arrays do seem to perform better for nsieve than other
sequenced collections.

If you have a fast list-based Erlang implementation for nsieve we'd be
happy to show it.




       
____________________________________________________________________________________
Yahoo! oneSearch: Finally, mobile search 
that gives answers, not web links. 
http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC


From alexander.lamb@REDACTED  Sat Oct  6 22:20:10 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Sat, 6 Oct 2007 22:20:10 +0200
Subject: [erlang-questions] Beginners question on pattern matching,
	thank you
In-Reply-To: <20071005163240.GA13214@localhost.localdomain>
References: <7FA46B0E-57EB-4DF2-B68E-4972A797978B@rodanotech.ch>
	<20071005163240.GA13214@localhost.localdomain>
Message-ID: <233187C0-F946-4DDC-9FD6-FCE225BDD63B@rodanotech.ch>

Yes, my mistake was obviously to mistake pattern matching with some  
king of regular expression handling.

Thanks a lot for your help.

Alex

Le 5 oct. 07 ? 18:32, Alexander Petry a ?crit :

> * Alexander Lamb <alexander.lamb@REDACTED> [071005 18:06]:
>> Hello List,
>>
>> I am starting on Erlang. I read most of the new Erlang book. However,
>> I am lost with something really simple and probably stupid.
>>
>> I am reading a file, line by line.
>>
>> Each line is in the form
>>
>> "a_code" , a_date
>>
>> The quotes (") are present in the file, as well as the comma. For
>> example:
>>
>> "C51CBA979E4311D67900201842D2EEE81",2003-08-07
>>
>> I would like to cut the string into pieces (to be able to modify
>> it...). Again, this is a test to learn Erlang, I know how to do this
>> in two lines of Ruby :-). I could also probably use a built in
>> function to parse the string. But I would like to understand pattern
>> matching and lists.
>>
>> So I wrote this, first read the file:
>>
>> start() ->
>>
>> 	case file:open("ENRL_ENROLLMENT.csv",read) of
>>
>> 		{ok, S} 	->
>> 				Val = do_read(S),
>> 				file:close(S),
>> 				Result = process_lines(Val),
>> 				{ok,Result};
>>
>> 		{error, Why} 	->
>>
>> 				{error, Why}
>>
>> 	end.
>>
>> Then the lines:
>>
>> do_read(S) ->
>>
>> 	case io:get_line(S,'') of
>>
>> 		eof		-> [];
>> 		Line 		-> [Line | do_read(S)]
>>
>> 	end.
>>
>> Then process each line in the list of lines:
>>
>> process_lines(Lines) ->
>>
>> 	[Date || [_Code | "," ++ Date] <- Lines].
>
> I am new to Erlang, too, but i guess that _Code does only match a  
> single
> element of the list, i.e in your case the first ". A simple  
> solution would be
> to split up a single line like:
>
> 	[_Code, Date] = string:split(Line, ","),
>
> Regards,
> Alex, too ;-)
>

--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071006/a56901a3/attachment.htm>

From gbulmer@REDACTED  Sat Oct  6 01:55:18 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sat, 6 Oct 2007 01:55:18 +0200
Subject: [erlang-questions] Attracting Functional Programmers
Message-ID: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>

During the CUFP workshop, people talked about attracting developers  
to FP.
The Haskell web site was compared to the Python (I think) web site.
I believe the pitch for Haskell was much weaker than the Python pitch.

I took a stab at a few 'marketing' statements for Haskell. My  
intention is to stimulate debate because I do feel there are  
unnecessary barriers to recruitment. I do not expect my suggestion to  
survive :-)

You may want to take a look at http://groups.google.com/group/cufp/ 
topics
I am G B-)

I don't feel I should stimulate a similar debate about the Erlang web  
site, which I think is different (but I don't mind trying; I've had  
bricks thrown at me before).
I do feel there should be some commonality of key ideas/benefits  
across the FP community where practical, so that the strength and  
coherence of the whole community and its technology is appreciated  
rather than highlighting a specific weaknesses of any individual part.

G Bulmer

PS - Great Erlang meeting; thank you everyone for the welcome,  
enthusiasm and support.


From bhatti_shahzad@REDACTED  Sun Oct  7 01:55:39 2007
From: bhatti_shahzad@REDACTED (shahzad bhatti)
Date: Sat, 6 Oct 2007 16:55:39 -0700 (PDT)
Subject: [erlang-questions] Stateful gen_servers
Message-ID: <317260.66052.qm@web81109.mail.mud.yahoo.com>

I would like to use gen_server to develop a stateful server, and since gen_servers are  generally communicated using module:function(arg) syntax, what's the best way to use them as stateful servers per client, i.e., each for each client a gen_server is started and is terminated when client asks explicitly or when client is terminated. Thanks.
   

       
---------------------------------
Building a website is a piece of cake. 
Yahoo! Small Business gives you all the tools to get online.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071006/10db0867/attachment.htm>

From dking@REDACTED  Sun Oct  7 03:45:17 2007
From: dking@REDACTED (David King)
Date: Sat, 6 Oct 2007 18:45:17 -0700
Subject: [erlang-questions] Stateful gen_servers
In-Reply-To: <317260.66052.qm@web81109.mail.mud.yahoo.com>
References: <317260.66052.qm@web81109.mail.mud.yahoo.com>
Message-ID: <C38F7BCE-AEA4-41FB-81D2-3E68BBE94EA5@ketralnis.com>

> I would like to use gen_server to develop a stateful server, and  
> since gen_servers are  generally communicated using module:function 
> (arg) syntax, what's the best way to use them as stateful servers  
> per client, i.e., each for each client a gen_server is started and  
> is terminated when client asks explicitly or when client is  
> terminated. Thanks.

You could have each client do something like:

Session=my_server:logon(),
Cookie=my_server:get_cookie(Session),
my_server:eat_cookie(Session,Cookie), [...]

Then in the server, the State field could have a dict of Sessions  
(generated via make_ref()?) to state information

If that's not enough, I have a tictactoe server that I wrote recently  
that maintains the states of games this way, so let me know if you  
want some live code that does this


From jay@REDACTED  Sun Oct  7 04:45:48 2007
From: jay@REDACTED (Jay Nelson)
Date: Sat, 6 Oct 2007 19:45:48 -0700
Subject: [erlang-questions] Code vs. binary heap storage
Message-ID: <B73FAC3A-D103-479B-A430-209ACC1659BE@duomark.com>

Given the following code:

return_bin(pinocchio) ->  <<"Once upon a time... ">>;
return_bin(cinderella) -> <<"In a castle long ago...">>;
return_bin(star_wars) -> <<"In a galaxy far, far way...">>.

Assume each of the binaries is of some significant size > 32 bytes.


If the module containing this function is code-loaded into a node, I  
want to verify whether the following are true:

1) The binary only exists once regardless of how many processes or  
modules access the functions.
2) The binary for each branch of the function is stored in the binary  
heap.
3) The function return is a pointer to the existing binary heap element.
4) The memory footprint of a process which can call these functions  
does not include the size of the binaries, even if the functions are  
called and a process variable is bound to the value.
5) These binaries can never be garbage collected unless the module  
containing the functions is unloaded (and all other references  
obtained by calling the functions are released).
6) Having a process per binary and a process to route requests would  
not be more or less memory efficient than having a single process  
with the above function in place of the routing process.

jay



From bhatti_shahzad@REDACTED  Sun Oct  7 06:23:56 2007
From: bhatti_shahzad@REDACTED (shahzad bhatti)
Date: Sat, 6 Oct 2007 21:23:56 -0700 (PDT)
Subject: [erlang-questions] Stateful gen_servers
In-Reply-To: <C38F7BCE-AEA4-41FB-81D2-3E68BBE94EA5@ketralnis.com>
Message-ID: <49622.97656.qm@web81106.mail.mud.yahoo.com>

Thanks David for the tip. I think I can use session-id to track state and have the client pass it with each request. Though, I would liked it if somehow that session-id is implicitly passed. Also, it would help if you can share a sample application.

David King <dking@REDACTED> wrote: > I would like to use gen_server to develop a stateful server, and  
> since gen_servers are  generally communicated using module:function 
> (arg) syntax, what's the best way to use them as stateful servers  
> per client, i.e., each for each client a gen_server is started and  
> is terminated when client asks explicitly or when client is  
> terminated. Thanks.

You could have each client do something like:

Session=my_server:logon(),
Cookie=my_server:get_cookie(Session),
my_server:eat_cookie(Session,Cookie), [...]

Then in the server, the State field could have a dict of Sessions  
(generated via make_ref()?) to state information

If that's not enough, I have a tictactoe server that I wrote recently  
that maintains the states of games this way, so let me know if you  
want some live code that does this


       
---------------------------------
Moody friends. Drama queens. Your life? Nope! - their life, your story.
 Play Sims Stories at Yahoo! Games. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071006/7f8ef926/attachment.htm>

From dking@REDACTED  Sun Oct  7 07:28:39 2007
From: dking@REDACTED (David King)
Date: Sat, 6 Oct 2007 22:28:39 -0700
Subject: [erlang-questions] Stateful gen_servers
In-Reply-To: <49622.97656.qm@web81106.mail.mud.yahoo.com>
References: <49622.97656.qm@web81106.mail.mud.yahoo.com>
Message-ID: <F7918155-F04D-4EE5-8929-FC999B3DEC69@ketralnis.com>

>>> I would like to use gen_server to develop a stateful server
>> You could have each client do something like: [...]
> Thanks David for the tip. I think I can use session-id to track  
> state and have the client pass it with each request. Though, I  
> would liked it if somehow that session-id is implicitly passed.

You could have the session implicitly passed by storing it in the  
process dictionary, like this:

-module(my_server).
-behaviour(gen_server).

[gen_server stuff...]

login() ->
   Session=gen_server:call(?MODULE,login),
   put(my_server_session_id,session),
   Session.

eat_cookies() ->
   case get(my_server_session_id) of
     undefined -> {error,not_logged_in};
     Session ->
       gen_server:call(?MODULE,{eat_cookies,Session})
   end.

eat_cookies(Session) ->
   gen_server:call(?MODULE,{eat_cookies,Session}).

There are two versions of eat_cookies so that you can explicitly  
specify a session if you need to, like if you need to use a session  
from a different process than that created it. Generally speaking,  
using the process dictionary is frowned upon, but this is how you'd  
do it if you wanted to use implicit sessions (see <http:// 
www.erlang.org/course/advanced.html#dict>).

> Also, it would help if you can share a sample application.

Sure, here's an example: <http://www.ketralnis.com/xythoswfs/webui/ 
_xy-4309_1-t_IwEbcrLc> (That link will remain valid for a few days at  
most.)

That's a tarball of the tictactoe server. I wrote that as an example  
for someone, so it's not made for easy starting, just easy reading,  
but you probably don't need to start it. The gen_server that you're  
looking for is src/board_server.erl, which is effectively just a  
state-server to keep track of the boards. It also includes  
rudimentary garbage collection for those sessions

Please feel free to comment on the code, I promise I won't be  
offended, and everybody needs a good code-review :) You can read,  
modify and keep it around, but please don't re-distribute it in any  
way. I'll describe it in an article as an example application after  
it's served its offline purpose, and I'll release the code then with  
an actual licence, but this isn't final, and it doesn't have a  
licence or anything, so please don't re-distribute it.

To actually run it, you'd have to compile Erlyweb (it's in the  
tarball in lib/), modify paths in various files (mostly start.sh and  
yaws.conf, but there may be others), run start.sh, and copy-paste the  
lines that it echoes before it starts yaws into the yaws prompt. Then  
connect to <http://localhost:4000/tictactoe>



From davidnwelton@REDACTED  Sun Oct  7 10:37:36 2007
From: davidnwelton@REDACTED (David Welton)
Date: Sun, 7 Oct 2007 10:37:36 +0200
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
Message-ID: <9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>

On 10/6/07, G Bulmer <gbulmer@REDACTED> wrote:
> During the CUFP workshop, people talked about attracting developers
> to FP.
> The Haskell web site was compared to the Python (I think) web site.
> I believe the pitch for Haskell was much weaker than the Python pitch.
>
> I took a stab at a few 'marketing' statements for Haskell. My
> intention is to stimulate debate because I do feel there are
> unnecessary barriers to recruitment. I do not expect my suggestion to
> survive :-)
>
> You may want to take a look at http://groups.google.com/group/cufp/
> topics
> I am G B-)

I think it's a fascinating subject.  If you don't mind my trying to
foist off some of my own work on you, I've written about it a little
bit.

http://www.welton.it/articles/programming_language_economics.html

My take is that if you introduce a new language, and you don't have
millions of dollars to push it (Java, C#), you're going to have to
find some niche, and do it an order of magnitude better than the
competition.  Something that's not just an incremental improvement,
but a clearly superior way of doing things, where "things" means
something that matters to your average industry programmer.

To my way of thinking, this would explain why Erlang is getting a lot
of attention from people these days - it does concurrency really well.
 That one thing, which is seen as increasingly important in a
multi-core future, is enough for people to pick up Erlang and look at
it.  In the past, Tk was enough to get people interested in Tcl, the
promise of very easy dynamic web pages got people interested in PHP,
Ruby on Rails got people interested in Ruby, and so on.  Of course,
that's not the only way languages grow in popularity - Python
gradually built up a following by being simple, clean and practical
for a wide variety of tasks.  It does seem to help a lot though - if
you get that "killer app", people will, at least for a while, ignore
other shortcomings in a language, which is crucial, because it's very
likely that a newcomer won't have huge libraries, books, user groups
and some of the other positive externalities that come with
popularity.

-- 
David N. Welton
http://www.welton.it/davidw/


From erlang@REDACTED  Sun Oct  7 12:06:01 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Sun, 7 Oct 2007 12:06:01 +0200
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
Message-ID: <9b08084c0710070306j457cb5a7hbb5ad2a934d16098@mail.gmail.com>

I think we have to answer the question:

    "what is the compelling reason to use Erlang?"

Its not ('cos it's FP)

The  2 second pitch which I used at JAOO was

Erlang = Parallel programming without pain

I'm not sure if this is best - it could be

Fault-tolerant ptogramming without pain or

Multicore programming wthout pain

I hope this answers the question

/Joe


On 10/7/07, David Welton <davidnwelton@REDACTED> wrote:
> On 10/6/07, G Bulmer <gbulmer@REDACTED> wrote:
> > During the CUFP workshop, people talked about attracting developers
> > to FP.
> > The Haskell web site was compared to the Python (I think) web site.
> > I believe the pitch for Haskell was much weaker than the Python pitch.
> >
> > I took a stab at a few 'marketing' statements for Haskell. My
> > intention is to stimulate debate because I do feel there are
> > unnecessary barriers to recruitment. I do not expect my suggestion to
> > survive :-)
> >
> > You may want to take a look at http://groups.google.com/group/cufp/
> > topics
> > I am G B-)
>
> I think it's a fascinating subject.  If you don't mind my trying to
> foist off some of my own work on you, I've written about it a little
> bit.
>
> http://www.welton.it/articles/programming_language_economics.html
>
> My take is that if you introduce a new language, and you don't have
> millions of dollars to push it (Java, C#), you're going to have to
> find some niche, and do it an order of magnitude better than the
> competition.  Something that's not just an incremental improvement,
> but a clearly superior way of doing things, where "things" means
> something that matters to your average industry programmer.
>
> To my way of thinking, this would explain why Erlang is getting a lot
> of attention from people these days - it does concurrency really well.
>  That one thing, which is seen as increasingly important in a
> multi-core future, is enough for people to pick up Erlang and look at
> it.  In the past, Tk was enough to get people interested in Tcl, the
> promise of very easy dynamic web pages got people interested in PHP,
> Ruby on Rails got people interested in Ruby, and so on.  Of course,
> that's not the only way languages grow in popularity - Python
> gradually built up a following by being simple, clean and practical
> for a wide variety of tasks.  It does seem to help a lot though - if
> you get that "killer app", people will, at least for a while, ignore
> other shortcomings in a language, which is crucial, because it's very
> likely that a newcomer won't have huge libraries, books, user groups
> and some of the other positive externalities that come with
> popularity.
>
>
> --
> David N. Welton
> http://www.welton.it/davidw/
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ulf@REDACTED  Sun Oct  7 13:41:00 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sun, 7 Oct 2007 13:41:00 +0200
Subject: [erlang-questions] Stateful gen_servers
In-Reply-To: <F7918155-F04D-4EE5-8929-FC999B3DEC69@ketralnis.com>
References: <49622.97656.qm@web81106.mail.mud.yahoo.com>
	<F7918155-F04D-4EE5-8929-FC999B3DEC69@ketralnis.com>
Message-ID: <8209f740710070441o7e2e8a24k5db9e5a2ab9dcd22@mail.gmail.com>

2007/10/7, David King <dking@REDACTED>:
> >>> I would like to use gen_server to develop a stateful server
> >> You could have each client do something like: [...]
> > Thanks David for the tip. I think I can use session-id to track
> > state and have the client pass it with each request. Though, I
> > would liked it if somehow that session-id is implicitly passed.
>
> You could have the session implicitly passed by storing it in the
> process dictionary, like this:
>
> -module(my_server).
> -behaviour(gen_server).
>
> [gen_server stuff...]
>
> login() ->
>    Session=gen_server:call(?MODULE,login),
>    put(my_server_session_id,session),
>    Session.

I tend to want the server to keep track of client pids, rather than
having the client pass an identifier in each call. Not that it
matters much in terms of security (since there is no way to
prevent spoofing anyway), but it _feels_ better.  (:

The server will verify that the client pid is logged in, and will send
the reply only to that pid.

Another trick, which is more of a therapeutic one (again, since we
have no real security), is to verify that the caller has actually entered
through one of the exported interface functions:

call(Server, Request) ->
   MRef = erlang:monitor(process, Server),
   Server ! {call, self(), MRef, Request},
   my_await_reply(MRef).

my_await_reply(MRef) ->
   receive
      {MRef, Reply} -> Reply;
      ...
   end.

and then, on the server side:

server_loop(St) ->
   receive
      {From, Ref, Request} ->
         case valid_request(From) andalso logged_in(From, St) of
             true ->
                {Reply, St1} = handle_request(Ref, Request),
                From ! {Ref, Reply},
                server_loop(St1);
             false ->
                %% ignore, or possibly send an error reply
                 server_loop(St)
         end;
       ...
   end.

valid_request(Pid) ->
   case process_info(Pid, current_function) of
      {current_function, {?MODULE,my_await_reply,1}} ->
         true;
      _ ->
         false
   end.

(assuming that my_await_reply/1 is not exported, and a few other things,
this is a cheap way of preventing someone from faking a request, by
just sending a message).

BR,
Ulf W


From ulf@REDACTED  Sun Oct  7 14:23:42 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sun, 7 Oct 2007 14:23:42 +0200
Subject: [erlang-questions] erlocaml
Message-ID: <8209f740710070523w6427d8b2se2e28df75dc8bd86@mail.gmail.com>

I mentioned the ErlOCaml project on the Erlang Workshop
in Freiburg.  I thought I'd at least get the URLs right:

Google code project:
http://code.google.com/p/erlocaml/

Google group:
http://groups.google.com/group/erlocaml-discuss

Like I said in the Workshop, not much has happened yet.

BR,
Ulf W


From saleyn@REDACTED  Sun Oct  7 15:52:24 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Sun, 07 Oct 2007 08:52:24 -0500
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
	N} socket option?
Message-ID: <4708E498.50401@gmail.com>

I was playing with Scott's Unix Domain Socket driver from jungerl and 
ran into the following issue.

gen_tcp:connect/2, gen_tcp:listen/2 functions expose an option {fd, 
FileDescriptor} that allows to take an elsewhere opened file descriptor 
and make it usable in the gen_tcp module.  This is a very convenient 
option, however, what I discovered was that the default options that are 
being set on a socket by prim_inet:accept_opts/2 call involved in 
gen_tcp:accept/1 try to get/set a 'priority' option that may not be 
supported

Here's the synopsis using unixdom_drv:

1> {File, Server} = {"/tmp/sock", 1}.
2> {ok, Port} = unixdom_drv:start().
3> {ok, US} = unixdom_drv:open(Port, File, Server).
4> {ok, FD} = unixdom_drv:getfd(Port, US).
5> {ok, Sock} = gen_tcp:listen(0, [binary,{packet,2},{fd,FD}).
6> gen_tcp:accept(Sock).
[... In another terminal window do: unixdom_drv:open(Port, File, 0) ...]
{error, einval}

The root of the issue is here (that is the call made by prim_inet right 
after successful async_accept):

7> prim_inet:getopts(Sock, [priority]).
{priority,0}
8> prim_inet:setopts(Sock, [{priority,0}]).
{error, einval}

Apparently unix domain sockets don't support this option.  I would think 
that one workaround would be to clear this priority option on the 
listening socket prior to calling gen_tcp:accept/1, but I don't see a 
way of doing so.

It seems to me that the list of default options in 
prim_inet:accept_opts/2 needs to be able to be customized by the caller 
somehow.

Any suggestions?

Serge


From gbulmer@REDACTED  Sun Oct  7 17:14:43 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 17:14:43 +0200
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
Message-ID: <BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com>

David

Interesting points. While I agree with most of your points, I think  
there is a fairly well understood model which seeks to address and  
organise a broad picture for technology adoption. A key approach is  
to focus on things that the adopter will recognise. I don't feel  
these are economic (in it's simple sense), or even very technical.

First, let me say that I believe Erlang/OTP has 3 key 'sweet spots'
1. Highly available, robust, systems. We can economically build  
Erlang systems which can provide extreme availability, deal with  
failure and 'heal' in a much wider range of circumstances than any  
economically viable alternative. Why would anyone *ever* build a  
system with inferior availability? I don't think it's simple economic  
grounds, because Erlang/OTP technology is cost competitive , and  
there is some evidence (unlike many other 'productivity  
technologies') that its productivity can offset "ramp-up costs" and/ 
or higher salaries.
2. Scaleability, Erlng/OTP can scale to support every size of  
enterprise, utility, national and international system. I think you  
could run the next Olymics on Erlang/OTP. Or the UK National Health  
service. Property 1 makes Erlang/OTP compelling. Again, I don't think  
it's the economics of the technology which is preventing this.
3*. Taking advantage of many-core processors. I believe there are  
many different types of problems/ Erlang/OTP is well suited to, from  
small grained problems upwards. Once more, though, I don't feel it is  
the economics of the technology which needs to be overcome.

Summary, in these three areas, Erlang is the most mature and  
commercially viable technology available.
IMHO, it's that simple.

I can assert that because if there was a better technology, we would  
all know about it (and I wouldn't be here !-)

To some extent, these are independent, and so could be pursued  
differently. I think 3 becomes 'killer' as numbers of cores  
increases, but today, 1 and 2 are, IMHO 'killer' if we can overcome  
the actual obstacles. I do not believe the obstacles are economic

I am a fan of "Crossing the Chasm:
Marketing and selling technology products to mainstream customers:  
Marketing and Selling Technology Products to Mainstream Customers"
by Geoffrey A. Moore
http://www.amazon.co.uk/Crossing-Chasm-technology-mainstream- 
Technology/dp/1841120634/ref=pd_bbs_sr_1/202-3897543-8546206? 
ie=UTF8&s=books&qid=1191761111&sr=8-1
(and his other book, Riding the Tornado)

This provides a rich model of technology adoption from creation,  
through visionaries, early adopters, early majority, main stream and  
eventually luddites (yes the day may eventually come when people  
think Erlang/OTP is very old, and the world has moved on. I vaguely  
recall, when designing Ada, the DoD thought 40 years was the  
*minimum* useful life for their purposes). I you are interested in  
this area, and haven't read it, I strongly recommend you do. I think  
he covers all of David's points, in the more general case, along with  
some evidence.


A key message of Geoffrey Moore is identify a target market, and  
focus on the early adopters (I think Erlang is well past the  
visionary stage). This largely means adjusting the product to fit the  
key needs of a specific market (preferably specific customers). If  
I've understood your points David, this is what you believe too. I  
believe Erlang/OTP has already done this for telecoms, but questions  
remains as to whether there is more to be done for other markets. For  
example, I believe that a few more pieces would make Erlang much more  
attractive to manufacturing industries. Similarly, a little work to  
enable 'rich client' (AKA AJAX-style) web applications, etc. I accept  
that, to experienced Erlang-ers these things may be no more than  
hours, or a few weeks, but, you aren't the adopter. For *potential  
adopters* it may seem a very far away goal.

Moore goes onto explain, once you've got the early adopters, go after  
the "more aware" members of the target market sector, the early  
majority. To capture the "early majority"; resolve their issues by  
make the product attractive to them. This is usually not technology  
or functionality issues. I think the target users in this group are  
companies; developers are often visionaries and early adopters. For  
companies, it is about forms of risk, credibility and publicity.  
Ericsson and a few others are the early adopters, and the detail of  
their use and experience is critical collateral for taking Erlang  
further to the early majority. The purpose of this collateral is to  
reassure potential adopters that the technology is low-risk; that it  
works well enough that they "won't get fired for buying in to it",  
and it has a lifetime. I believe Open Sourcing helps, but, there are  
many, many more dead Open Source projects than live ones; the vibrant  
community around Erlang is very important too.

But key, is the evidence from actual users, who deploy it at similar  
or better levels of scale, availability and value to their business,  
because two key benefit are deploying Erlang/OTP at scale in highly- 
available applications. This evidence doesn't come from inspection of  
the functionality, or by designing and running benchmarks, or  
anything else that a a small group of people could complete  
internally in a couple of months feasibility study.

When a prospective adopter looks at Erlang they need evidence. As an  
example, imagine I was some company reading about Erlang with a view  
to potentially using it. I could read that Ericsson uses Erlang in a  
product, and that would help demonstrate that it has got beyond the  
lab. Unfortunately I can't directly find out if the product is  
successful, has been widely purchased, is it still in production, or  
whether it is in a niche used by one customer who has subsequently  
given up using it. IMHO succesfully marketing technologies does not  
require 'rocket science', just making sure the right information is  
easily available.

Other valuable evidence for  prospective adopter are things like  
support cost, etc, etc. Some of these are commercialy sensitive, and  
we may need to find ways to keep everyone happy.

For any adopter, a key issue is "how do I get support?" For example,  
what would happen to his notional adopter if they adopted Erlang/OTP  
for one project, but went no further? A bad answer is "that would be  
a career or economic disaster"! As an adopter, I would not want the  
burden of retaining an in-house team of Erlang/OTP experts who are so  
familiar with Erlang/OTP's internals that they could fix any problem.  
I would want to know how I get support from a 3rd party, at a  
reasonable price, and likely to be there whan I need them for the  
forceable future. I believe Ericsson handle this through their  
commercial license, but that may mean I couldn't experiment with a  
smaller, lower cost, potentially lower risk, project on Open Source  
Erlang/OTP first. Being forced to *START* with commercial Erlang/OTP  
makes adoption more complex, expensive and less likely.

The Erlang community could deal with this. Maybe it could offer a  
service, maybe through Erlang.org, whereby a customer could buy a  
'bug fix'. [Microsoft sell 'an incident' where you can buy a fix to a  
problem for approximately $100, but lets not get side tracked onto  
them.]

None of this is to say that *I* am unconvinced. After meeting the  
folks who build Erlang, I am much, much happier. The point I am  
trying to make is, I shouldn't need to stake my reputation and  
judgement when proposing Erlang in a project. The evidence should be  
available so that the decision process to choose Erlang is rationale  
and transparent, and the cost and risks are easily and clearly  
identifiable upfront, with no surprises.

So, I think Erlang/OTP adoption would be significantly improved by  
gathering and organising up evidence of its successful and continued  
use, and more simply and clearly defining the markets in which it is  
a 'good fit'. It is useful to define the markets, and not just the  
properties of the applications, so that potential adopters can  
recognise that it is already ready for them.

HTH
G Bulmer

PS * - I would like to qualify point 3, about Erlang and many-core.  
There are applications, where alternatives are very relevant too. For  
example image processing using photoshop or the gimp, where even  
finer grained data-parallel technology is extremly effective. So,  
though it is completely practical to write a gimp plugin for image  
processing with Erlang/OTP (I have written one using NVIDIA CUDA, so  
I can speak from experience), technologies which implement even finer  
grained parallelism are also relevant. But this isn't to detract from  
the point, just remind us that it isn't necessarily one size fits all.


> On 10/6/07, G Bulmer <gbulmer@REDACTED> wrote:
>> During the CUFP workshop, people talked about attracting developers
>> to FP.
>> The Haskell web site was compared to the Python (I think) web site.
>> I believe the pitch for Haskell was much weaker than the Python  
>> pitch.
>>
>> I took a stab at a few 'marketing' statements for Haskell. My
>> intention is to stimulate debate because I do feel there are
>> unnecessary barriers to recruitment. I do not expect my suggestion to
>> survive :-)
>>
>> You may want to take a look at http://groups.google.com/group/cufp/
>> topics
>> I am G B-)
>
> I think it's a fascinating subject.  If you don't mind my trying to
> foist off some of my own work on you, I've written about it a little
> bit.
>
> http://www.welton.it/articles/programming_language_economics.html
>
> My take is that if you introduce a new language, and you don't have
> millions of dollars to push it (Java, C#), you're going to have to
> find some niche, and do it an order of magnitude better than the
> competition.  Something that's not just an incremental improvement,
> but a clearly superior way of doing things, where "things" means
> something that matters to your average industry programmer.
>
> To my way of thinking, this would explain why Erlang is getting a lot
> of attention from people these days - it does concurrency really well.
>  That one thing, which is seen as increasingly important in a
> multi-core future, is enough for people to pick up Erlang and look at
> it.  In the past, Tk was enough to get people interested in Tcl, the
> promise of very easy dynamic web pages got people interested in PHP,
> Ruby on Rails got people interested in Ruby, and so on.  Of course,
> that's not the only way languages grow in popularity - Python
> gradually built up a following by being simple, clean and practical
> for a wide variety of tasks.  It does seem to help a lot though - if
> you get that "killer app", people will, at least for a while, ignore
> other shortcomings in a language, which is crucial, because it's very
> likely that a newcomer won't have huge libraries, books, user groups
> and some of the other positive externalities that come with
> popularity.
>
> -- 
> David N. Welton
> http://www.welton.it/davidw/



From harveyd@REDACTED  Sun Oct  7 17:37:32 2007
From: harveyd@REDACTED (Dale Harvey)
Date: Sun, 7 Oct 2007 16:37:32 +0100
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
	<BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com>
Message-ID: <fc6cbe440710070837l1c49c6b9y445df9a64a9ee5b7@mail.gmail.com>

In terms of attracting programmers, it seems like my main reason, if
noticing the traction that erlang has been gaining recently, to not decided
to give erlang a shot is the fact that I would have to own my own server in
order to develop anything public in it.

Personally I have tested quite a few web based languages, all of which were
little experiments which I could 'deploy' on my shared hosting. If it werent
for other circumstances I almost certainly wouldnt have tested erlang.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071007/0de1ce1a/attachment.htm>

From per@REDACTED  Sun Oct  7 18:09:02 2007
From: per@REDACTED (Per Hedeland)
Date: Sun, 7 Oct 2007 18:09:02 +0200 (CEST)
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
	N} socket option?
In-Reply-To: <4708E498.50401@gmail.com>
Message-ID: <200710071609.l97G92Gd085783@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>gen_tcp:connect/2, gen_tcp:listen/2 functions expose an option {fd, 
>FileDescriptor} that allows to take an elsewhere opened file descriptor 
>and make it usable in the gen_tcp module.  This is a very convenient 
>option, however, what I discovered was that the default options that are 
>being set on a socket by prim_inet:accept_opts/2 call involved in 
>gen_tcp:accept/1 try to get/set a 'priority' option that may not be 
>supported

Well, it's rather dubious to pass in a non-inet socket to the inet
modules/driver, but anyway the problem you're running into isn't really
that 'priority' isn't supported (SO_PRIORITY is a Linux-specific thing
AFAIK, but it's a socket-level option and so works for any type of
socket), but rather that the driver does some pretty ugly things in an
attempt to make 'priority' and 'tos' appear to be independent, even
though they aren't at the OS level (it's a mistake to do this I think,
but let's ignore that for the moment).

This has the effect that it ends up trying to set the IP_TOS option on
your unix-domain socket, which doesn't work of course, and then it gives
up on the whole thing even though it was actually SO_PRIORITY that was
requested (from the prim_inet code you quoted).

Below is a patch that addresses this - it's against R10B-10 though, and
won't apply against current versions since even though the code is
essentially the same there, it has moved around a bit. But maybe you can
convert it - or just comment out the whole setopt_prio_tos_trick() thing
there.:-)

I have subsequently hacked prim_inet and inet_drv to handle arbitrary
non-inet sockets "properly", i.e. allow for passing in 'unspec' (as in
AF_UNSPEC) as the address family, and in that case refrain from doing
*any* AF_INET/AF_INET6-specific get/setsockopts, as well as
getsockname() and getpeername() (since the address format is unknown).
It's perhaps still a bit dubious to "abuse" the inet code this way
though, but you sure get a lot of stuff for free that way...

--Per Hedeland

Index: otp/erts/emulator/drivers/common/inet_drv.c
===================================================================
--- otp/erts/emulator/drivers/common/inet_drv.c	(revision 11053)
+++ otp/erts/emulator/drivers/common/inet_drv.c	(revision 11062)
@@ -3856,6 +3856,15 @@
 	default:
 	    return -1;
 	}
+
+/* Per H @ Tail-f: The original code here had problems that possibly
+   only occur if you abuse it for non-INET sockets, but anyway:
+   a) If the getsockopt for SO_PRIORITY or IP_TOS failed, the actual
+      requested setsockopt was never even attempted.
+   b) If {get,set}sockopt for one of IP_TOS and SO_PRIORITY failed,
+      but ditto for the other worked and that was actually the requested
+      option, failure was still reported to erlang.                  */
+
 #if  defined(IP_TOS) && defined(SOL_IP) && defined(SO_PRIORITY)
 	{
 	    /* The relations between SO_PRIORITY, TOS and other options
@@ -3868,34 +3877,36 @@
 	       user feeling socket options are independent. /PaN */
 	    int tmp_ival_prio;
 	    int tmp_arg_sz_prio = sizeof(tmp_ival_prio);
+	    int res_prio;
 	    int tmp_ival_tos;
 	    int tmp_arg_sz_tos = sizeof(tmp_ival_tos);
+	    int res_tos;
 
-	    res = sock_getopt(desc->s, SOL_SOCKET, SO_PRIORITY,
-			      (char *) &tmp_ival_prio, &tmp_arg_sz_prio);
+	    res_prio = sock_getopt(desc->s, SOL_SOCKET, SO_PRIORITY,
+				   (char *) &tmp_ival_prio, &tmp_arg_sz_prio);
+	    res_tos = sock_getopt(desc->s, SOL_IP, IP_TOS, 
+				  (char *) &tmp_ival_tos, &tmp_arg_sz_tos);
+	    res = sock_setopt(desc->s, proto, type, arg_ptr, arg_sz);
 	    if (res == 0) {
-		res = sock_getopt(desc->s, SOL_IP, IP_TOS, 
-			      (char *) &tmp_ival_tos, &tmp_arg_sz_tos);
-		if (res == 0) {
-		    res = sock_setopt(desc->s, proto, type, arg_ptr, arg_sz);
-		    if (res == 0) {
-			if (type != SO_PRIORITY) {
-			    if (type != IP_TOS) {
-				res = sock_setopt(desc->s, 
-						  SOL_IP, 
-						  IP_TOS,
-						  (char *) &tmp_ival_tos, 
-						  tmp_arg_sz_tos);
-			    }
-			    if (res == 0) {
-				res =  sock_setopt(desc->s, 
-						   SOL_SOCKET, 
-						   SO_PRIORITY,
-						   (char *) &tmp_ival_prio, 
-						   tmp_arg_sz_prio);
-			    }
-			}
+		if (type != SO_PRIORITY) {
+		    if (type != IP_TOS && res_tos == 0) {
+			res_tos = sock_setopt(desc->s, 
+					      SOL_IP, 
+					      IP_TOS,
+					      (char *) &tmp_ival_tos, 
+					      tmp_arg_sz_tos);
+			if (propagate)
+			    res = res_tos;
 		    }
+		    if (res == 0 && res_prio == 0) {
+			res_prio = sock_setopt(desc->s, 
+					       SOL_SOCKET, 
+					       SO_PRIORITY,
+					       (char *) &tmp_ival_prio, 
+					       tmp_arg_sz_prio);
+			if (propagate)
+			    res = res_prio;
+		    }
 		}
 	    }
 	} 


From gbulmer@REDACTED  Sun Oct  7 19:36:23 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 19:36:23 +0200
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <fc6cbe440710070837l1c49c6b9y445df9a64a9ee5b7@mail.gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
	<BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com>
	<fc6cbe440710070837l1c49c6b9y445df9a64a9ee5b7@mail.gmail.com>
Message-ID: <890BC292-0A17-40A7-A9E6-6400FECCA1C2@gmail.com>


On 7 Oct 2007, at 17:37, Dale Harvey wrote:

> In terms of attracting programmers, it seems like my main reason,  
> if noticing the traction that erlang has been gaining recently, to  
> not decided to give erlang a shot is the fact that I would have to  
> own my own server in order to develop anything public in it.
>
> Personally I have tested quite a few web based languages, all of  
> which were little experiments which I could 'deploy' on my shared  
> hosting. If it werent for other circumstances I almost certainly  
> wouldnt have tested erlang.

I'm not sure I understand your point.
Are you saying Erlang/OTP will become much more useful to people like  
you when it is made available as a "shrink wrapped" platform by  
hosting companies?
If that's your point, I agree.

AFAIK, the only company offering (or planing to offer) hosted Erlang  
was joyent, and Joyant were expensive compared to the majority of  
hosting companies.
As a comparison, Railsplayground.com offers Ruby on Rails hosting  
starting at $5/month. You just upload your app., and go.

The cheapest way for a company to offer Erlang hosting would be  
within virtual private servers (VPS), e.g. Linux Virtual Machines  
running Erlang Nodes. I've googled around and can't find anyone doing  
that. Of course, you could build the virtual private server yourself,  
but that seems like significant effort which a hosting company could  
just 'solve' for people.
Is there a pre-built VPS image to take out all of the pain?

A nice solution would be to have a "pre-configured, templated image"  
of an Erlang environment and Erlang/OTP at Amazon Elastic Compute  
Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011
Amazon's computing cloud supports starting up and closing down upnew  
servers on-demand, you only pay for what you use. This would be very  
handy if your throughput requirements are very peaky.

GB



From gbulmer@REDACTED  Sun Oct  7 20:16:05 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 20:16:05 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
Message-ID: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>

After last weeks excellent Erlang workshop, I am thinking about going  
to the 13th International Erlang/OTP User Conference.
Unfortunately, I don't know where it is (more precisely than Stockholm).

Is it in a hotel or a conference centre?
Is there a hotel offering a good deal on accommodation? I notice that  
Hotell ?lvsj? was offering a discount last year.

I realise this is very likely too late for this year, but I will  
suggest it anyway.
It takes several days out of my schedule to go to a conference, and  
it's much easier to justify to myself when there are a couple of days  
of content. I would be very interested in a full day of training- 
orientated sessions towards "Erlang for those who have largely read  
and understood Joe's book". The things I feel I miss are the  
idiomatic uses of Erlang (like John Hughes' tiny [X || X>0]), and I'd  
love a 1/2+ day chunk of "Programming Distributed Erlang  
Applications: Pitfalls & Recipes" with hands on exercises. I suspect  
other newbies would too.

I wouldn't expect it for free, though I am price sensitive (this is  
my holiday, after all :-), maybe 116 ?/person, with a group of 20  
people? We bring our laptops with Erlang installed and ready to go,  
and the organiser provides a wireless network to distribute example  
code, slides and stuff. The whole thing could be the day after the  
user conference.

If Erlang takes off big, it will be almost essential for next year.  
So the opportunity is to beta-test the whole process on us this year :-)

I notice that last year, the conference was 2 days long. Does anyone  
know why it's only one day, the 8th, this year, or is there more  
announcements to come?

G Bulmer



From ulf@REDACTED  Sun Oct  7 20:40:11 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sun, 7 Oct 2007 20:40:11 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
In-Reply-To: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
Message-ID: <8209f740710071140n6b4a638alc7bf5e45414c3f72@mail.gmail.com>

It will be in the same place as last year:

http://hitta.se/SearchCombi.aspx?vad=&var=armborstv%e4gen+%e4lvsj%f6
(Approximately - you should go to the main entrance).

As to whether it will be one or two days, I have no clue.

BR,
Ulf W

2007/10/7, G Bulmer <gbulmer@REDACTED>:
> After last weeks excellent Erlang workshop, I am thinking about going
> to the 13th International Erlang/OTP User Conference.
> Unfortunately, I don't know where it is (more precisely than Stockholm).
>
> Is it in a hotel or a conference centre?
> Is there a hotel offering a good deal on accommodation? I notice that
> Hotell ?lvsj? was offering a discount last year.
>
> I realise this is very likely too late for this year, but I will
> suggest it anyway.
> It takes several days out of my schedule to go to a conference, and
> it's much easier to justify to myself when there are a couple of days
> of content. I would be very interested in a full day of training-
> orientated sessions towards "Erlang for those who have largely read
> and understood Joe's book". The things I feel I miss are the
> idiomatic uses of Erlang (like John Hughes' tiny [X || X>0]), and I'd
> love a 1/2+ day chunk of "Programming Distributed Erlang
> Applications: Pitfalls & Recipes" with hands on exercises. I suspect
> other newbies would too.
>
> I wouldn't expect it for free, though I am price sensitive (this is
> my holiday, after all :-), maybe 116 ?/person, with a group of 20
> people? We bring our laptops with Erlang installed and ready to go,
> and the organiser provides a wireless network to distribute example
> code, slides and stuff. The whole thing could be the day after the
> user conference.
>
> If Erlang takes off big, it will be almost essential for next year.
> So the opportunity is to beta-test the whole process on us this year :-)
>
> I notice that last year, the conference was 2 days long. Does anyone
> know why it's only one day, the 8th, this year, or is there more
> announcements to come?
>
> G Bulmer
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From gbulmer@REDACTED  Sun Oct  7 21:08:15 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 21:08:15 +0200
Subject: [erlang-questions] Specifying Type Contracts in Erlang
Message-ID: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>

I've been reading the paper "A language for specifying type contracts  
in Erlang and its interaction with success typing' that was presented  
at the Erlang workshop. I think this is a good, useful and valuable  
piece of work.

As far as I can see, there are only 2 pieces which are not covered by  
existing Erlang syntax:
1. a range of values, like (0..255), and (0..), and
2. "..." which seems to be interpreted as 0 or more.

IMHO, it would be a good idea to stay within Erlangs existing syntax,  
and avoid extensions.
Case one, could be dealt with using a sequence, seq(0,255) with a  
reserved value for infinity, maybe inf(). I'd be happy inventing a  
new function, it doesn't have to be seq().
Case two appears to be the same as _. The example was [char(), ...]  
for non-empty string, which could also be written [char()|_]. I'm  
happy to use other alternatives like a 'magic' function, e.g. tail().

What benefits do I think we'd get by staying within Erlang syntax?
1. There is little new to learn.
2. The parser for type contracts (type annotations) is already  
written. This lowers the bar for others to reuse the type annotation  
information. I believe there is already a well defined representation  
for parsed Erlang which may reduce the amount of work to reuse  
annotations even more.
3. I believe there is already a well-defined mechanism for saving  
parsed Erlang in Beam files, so a tool needn't deal with Erlang  
source at all.
4. Erlang syntax could be extended in the future, without having any  
syntax 'stolen' by type annotations.

It was pointed out in the workshop that the syntax was proposed by  
Philip Wadler and Simon Peyton Jones. I accept these guys understand  
more about types than I ever will. But, I don't think they are likely  
to confuse surface syntax with underlying semantics. The underlying  
semantics seem to be clear. I am willing to believe I have missed  
some subtlety, but otherwise staying within Erlangs existing syntax  
seems to have the most benefits.

If there is a lot of work which has been expended on building  
annotations with the original annotation syntax, then it would be  
worth building a tool to translate the ... and 1..n syntax. I assume  
it should be relatively straightforward to develop a pretty printer  
based on the existing type contract parser.

So that is my main point, but I started to think, if the type  
contract/annotation syntax is cast within the syntax of normal  
Erlang, then we can consider using it within normal Erlang code. I  
don't think it gives any more power than guards, but it would be a  
pretty unification.

So, we might consider taking a type annotation and using it directly,  
for example in patterns:
receive X when [char()|tail()], pos_integer() -> fun(X) end.
Or even:
Pid ! [char()|tail()], pos_integer().

Also, assuming type-name functions yielded well defined values,  
storing type annotations in ETS/DETS should be straightforward. That  
might be a nice base for nifty tools which would like to check type  
annotations, maybe between versions of code.

Summary: I think here are some genuine benefits for type contracts to  
stay within Erlang syntax.
G Bulmer



From gbulmer@REDACTED  Sun Oct  7 21:15:38 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 21:15:38 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
In-Reply-To: <8209f740710071140n6b4a638alc7bf5e45414c3f72@mail.gmail.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
	<8209f740710071140n6b4a638alc7bf5e45414c3f72@mail.gmail.com>
Message-ID: <5DE85FE2-0D98-49E2-B090-00857DCA9D99@gmail.com>

Thank you.

I have only been to Stockholm once before, for a morning meeting,  
after which I flew home, so I am thinking of staying on for a coupe  
of days.
Is this a pleasant part of the city to stay, or would you (or anyone)  
recommend some other part of the city?

G Bulmer


On 7 Oct 2007, at 20:40, Ulf Wiger wrote:

> It will be in the same place as last year:
>
> http://hitta.se/SearchCombi.aspx?vad=&var=armborstv%e4gen+%e4lvsj%f6
> (Approximately - you should go to the main entrance).
>
> As to whether it will be one or two days, I have no clue.
>
> BR,
> Ulf W
>
> 2007/10/7, G Bulmer <gbulmer@REDACTED>:
>> After last weeks excellent Erlang workshop, I am thinking about going
>> to the 13th International Erlang/OTP User Conference.
>> Unfortunately, I don't know where it is (more precisely than  
>> Stockholm).
>>
>> Is it in a hotel or a conference centre?
>> Is there a hotel offering a good deal on accommodation? I notice that
>> Hotell ?lvsj? was offering a discount last year.
>>
>> I realise this is very likely too late for this year, but I will
>> suggest it anyway.
>> It takes several days out of my schedule to go to a conference, and
>> it's much easier to justify to myself when there are a couple of days
>> of content. I would be very interested in a full day of training-
>> orientated sessions towards "Erlang for those who have largely read
>> and understood Joe's book". The things I feel I miss are the
>> idiomatic uses of Erlang (like John Hughes' tiny [X || X>0]), and I'd
>> love a 1/2+ day chunk of "Programming Distributed Erlang
>> Applications: Pitfalls & Recipes" with hands on exercises. I suspect
>> other newbies would too.
>>
>> I wouldn't expect it for free, though I am price sensitive (this is
>> my holiday, after all :-), maybe 116 ?/person, with a group of 20
>> people? We bring our laptops with Erlang installed and ready to go,
>> and the organiser provides a wireless network to distribute example
>> code, slides and stuff. The whole thing could be the day after the
>> user conference.
>>
>> If Erlang takes off big, it will be almost essential for next year.
>> So the opportunity is to beta-test the whole process on us this  
>> year :-)
>>
>> I notice that last year, the conference was 2 days long. Does anyone
>> know why it's only one day, the 8th, this year, or is there more
>> announcements to come?
>>
>> G Bulmer
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>



From alexander.lamb@REDACTED  Sun Oct  7 21:39:01 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Sun, 7 Oct 2007 21:39:01 +0200
Subject: [erlang-questions] Mnesia system events and table events
Message-ID: <FEAC09FC-8AE9-4427-A6B0-400BC07C3C0B@rodanotech.ch>

Hello,

I am trying to understand some Mnesia concepts and am wondering about  
the capabilities of notifying of changes.
Indeed, in a traditional application using a database such as MySQL  
for example, if I want to make sure I always have the last version of  
some data, I need to either:

A) never cache any data in memory and refetch data at each query
B) use a memory cache (we use Cayenne with Java) and make sure all  
reads and writes are always going through the same process / cache

In some situations this is not acceptable.

Therefore, in the situation were I implement some services using a  
database, I want to make sure that when the database changes, all  
processes (Erlang or others) get notified of a change.

To achieve this I thought it would be possible to implement the  
business logic  in Erlang using Mnesia as a back-end database. Then,  
have one Erlang process listening to all changes to the tables and  
for each change, send a message, for example on a message bus such as  
RabbitMQ. Then, clients in other languages than Erlang could listen  
to changes and update an object graph accordingly.

Is this the way to go? In details:

1) is using table events in Mnesia a usual way to notify other  
processes of changes?
2) I suppose the events are sent aynchronously... what about  
performance issues?
3) If I have a multi-node Mnesia database, I suppose I will only  
receive one event per event (not one per node)?
4) If this is the way to go, would you send on the bus:
	a) the Name and Key of the record(s) which changed (leaving the  
listening, non Erlang process to refetch immediately after what it  
needs)
	b) the complete record(s), event the attributes which didn't change
	c) only the changed attributes
I was thinking of converting the events in JSON messages which could  
be listened to from any non Erlang process. Actually, this seems  
rather generic so maybe someone already had a thought about that kind  
of architecture.

  In practice, we are considering rewriting a module of our system.  
By putting in place some standardized way of communicating between  
Applications we can develop the new system incrementally.

Thanks for any hints!

Alex
--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071007/0812f6b9/attachment.htm>

From qrilka@REDACTED  Sun Oct  7 21:41:55 2007
From: qrilka@REDACTED (Kirill Zaborski)
Date: Sun, 7 Oct 2007 23:41:55 +0400
Subject: [erlang-questions] Specifying Type Contracts in Erlang
In-Reply-To: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
References: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
Message-ID: <337538cb0710071241n7cf201b4w78a9aca57d540aa9@mail.gmail.com>

Is this paper publicly available somewhere? I think it would be interesting
to read.

Best regards,
Kirill.

On 10/7/07, G Bulmer <gbulmer@REDACTED> wrote:
>
> I've been reading the paper "A language for specifying type contracts
> in Erlang and its interaction with success typing' that was presented
> at the Erlang workshop. I think this is a good, useful and valuable
> piece of work.
>
> As far as I can see, there are only 2 pieces which are not covered by
> existing Erlang syntax:
> 1. a range of values, like (0..255), and (0..), and
> 2. "..." which seems to be interpreted as 0 or more.
>
> IMHO, it would be a good idea to stay within Erlangs existing syntax,
> and avoid extensions.
> Case one, could be dealt with using a sequence, seq(0,255) with a
> reserved value for infinity, maybe inf(). I'd be happy inventing a
> new function, it doesn't have to be seq().
> Case two appears to be the same as _. The example was [char(), ...]
> for non-empty string, which could also be written [char()|_]. I'm
> happy to use other alternatives like a 'magic' function, e.g. tail().
>
> What benefits do I think we'd get by staying within Erlang syntax?
> 1. There is little new to learn.
> 2. The parser for type contracts (type annotations) is already
> written. This lowers the bar for others to reuse the type annotation
> information. I believe there is already a well defined representation
> for parsed Erlang which may reduce the amount of work to reuse
> annotations even more.
> 3. I believe there is already a well-defined mechanism for saving
> parsed Erlang in Beam files, so a tool needn't deal with Erlang
> source at all.
> 4. Erlang syntax could be extended in the future, without having any
> syntax 'stolen' by type annotations.
>
> It was pointed out in the workshop that the syntax was proposed by
> Philip Wadler and Simon Peyton Jones. I accept these guys understand
> more about types than I ever will. But, I don't think they are likely
> to confuse surface syntax with underlying semantics. The underlying
> semantics seem to be clear. I am willing to believe I have missed
> some subtlety, but otherwise staying within Erlangs existing syntax
> seems to have the most benefits.
>
> If there is a lot of work which has been expended on building
> annotations with the original annotation syntax, then it would be
> worth building a tool to translate the ... and 1..n syntax. I assume
> it should be relatively straightforward to develop a pretty printer
> based on the existing type contract parser.
>
> So that is my main point, but I started to think, if the type
> contract/annotation syntax is cast within the syntax of normal
> Erlang, then we can consider using it within normal Erlang code. I
> don't think it gives any more power than guards, but it would be a
> pretty unification.
>
> So, we might consider taking a type annotation and using it directly,
> for example in patterns:
> receive X when [char()|tail()], pos_integer() -> fun(X) end.
> Or even:
> Pid ! [char()|tail()], pos_integer().
>
> Also, assuming type-name functions yielded well defined values,
> storing type annotations in ETS/DETS should be straightforward. That
> might be a nice base for nifty tools which would like to check type
> annotations, maybe between versions of code.
>
> Summary: I think here are some genuine benefits for type contracts to
> stay within Erlang syntax.
> G Bulmer
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071007/9ee12ebb/attachment.htm>

From kostis@REDACTED  Sun Oct  7 22:32:35 2007
From: kostis@REDACTED (Kostis Sagonas)
Date: Sun, 07 Oct 2007 23:32:35 +0300
Subject: [erlang-questions] Specifying Type Contracts in Erlang
In-Reply-To: <337538cb0710071241n7cf201b4w78a9aca57d540aa9@mail.gmail.com>
References: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
	<337538cb0710071241n7cf201b4w78a9aca57d540aa9@mail.gmail.com>
Message-ID: <47094263.9070908@cs.ntua.gr>

Kirill Zaborski wrote:
> Is this paper publicly available somewhere? I think it would be 
> interesting to read.

It is available at:

      http://user.it.uu.se/~kostis/Papers/contracts.pdf

Comments welcome.

Kostis


From richardc@REDACTED  Sun Oct  7 23:21:58 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Sun, 07 Oct 2007 23:21:58 +0200
Subject: [erlang-questions] Some pictures from CUFP and the Erlang workshop
Message-ID: <47094DF6.7000705@it.uu.se>

http://www.flickr.com/photos/rich_4711/sets/72157602309315717/

    /Richard


From bjarne@REDACTED  Sun Oct  7 23:42:32 2007
From: bjarne@REDACTED (=?Windows-1252?Q?Bjarne_D=E4cker?=)
Date: Sun, 7 Oct 2007 23:42:32 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP
	UserConference going to be?
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com><8209f740710071140n6b4a638alc7bf5e45414c3f72@mail.gmail.com>
	<5DE85FE2-0D98-49E2-B090-00857DCA9D99@gmail.com>
Message-ID: <004701c8092a$f9712f40$264bd854@Dell>

Hello

This year's EUC will take place in the same location as last year. 
Directions to the conference can be found in

http://www.erlang.se/euc/06/map.html

We published some hints last year too which we might update

http://www.erlang.se/euc/06/hints.html

Hotel ?lvsj? offers a rebate for EUC visitors during 7-9 November. Don't 
forget to mention "Erlang".

http://www.hotellalvsjo.nu/eng/index.htm

Last year we organised some seminars on the day after the EUC. We might do 
that again if somebody offers to present some subject or lead a discussion.

To register for the EUC just send a mail stating name and affiliation to 
euc@REDACTED

The last few years we have organised a little tour around Stockholm on the 
day before the EUC for foreign visitors ending up in some suitable pub.

Welcome !

Bjarne D?cker

EUC chairman joint with Kenneth Lundin

----- Original Message ----- 
From: "G Bulmer" <gbulmer@REDACTED>
To: "Ulf Wiger" <ulf@REDACTED>
Cc: "Erlang-Questions (E-mail)" <erlang-questions@REDACTED>
Sent: Sunday, October 07, 2007 9:15 PM
Subject: Re: [erlang-questions] Where is 13th International Erlang/OTP 
UserConference going to be?


Thank you.

I have only been to Stockholm once before, for a morning meeting,
after which I flew home, so I am thinking of staying on for a coupe
of days.
Is this a pleasant part of the city to stay, or would you (or anyone)
recommend some other part of the city?

G Bulmer


On 7 Oct 2007, at 20:40, Ulf Wiger wrote:

> It will be in the same place as last year:
>
> http://hitta.se/SearchCombi.aspx?vad=&var=armborstv%e4gen+%e4lvsj%f6
> (Approximately - you should go to the main entrance).
>
> As to whether it will be one or two days, I have no clue.
>
> BR,
> Ulf W
>
> 2007/10/7, G Bulmer <gbulmer@REDACTED>:
>> After last weeks excellent Erlang workshop, I am thinking about going
>> to the 13th International Erlang/OTP User Conference.
>> Unfortunately, I don't know where it is (more precisely than
>> Stockholm).
>>
>> Is it in a hotel or a conference centre?
>> Is there a hotel offering a good deal on accommodation? I notice that
>> Hotell ?lvsj? was offering a discount last year.
>>
>> I realise this is very likely too late for this year, but I will
>> suggest it anyway.
>> It takes several days out of my schedule to go to a conference, and
>> it's much easier to justify to myself when there are a couple of days
>> of content. I would be very interested in a full day of training-
>> orientated sessions towards "Erlang for those who have largely read
>> and understood Joe's book". The things I feel I miss are the
>> idiomatic uses of Erlang (like John Hughes' tiny [X || X>0]), and I'd
>> love a 1/2+ day chunk of "Programming Distributed Erlang
>> Applications: Pitfalls & Recipes" with hands on exercises. I suspect
>> other newbies would too.
>>
>> I wouldn't expect it for free, though I am price sensitive (this is
>> my holiday, after all :-), maybe 116 ?/person, with a group of 20
>> people? We bring our laptops with Erlang installed and ready to go,
>> and the organiser provides a wireless network to distribute example
>> code, slides and stuff. The whole thing could be the day after the
>> user conference.
>>
>> If Erlang takes off big, it will be almost essential for next year.
>> So the opportunity is to beta-test the whole process on us this
>> year :-)
>>
>> I notice that last year, the conference was 2 days long. Does anyone
>> know why it's only one day, the 8th, this year, or is there more
>> announcements to come?
>>
>> G Bulmer
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions 



From richardc@REDACTED  Sun Oct  7 23:54:45 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Sun, 07 Oct 2007 23:54:45 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
 Conference going to be?
In-Reply-To: <5DE85FE2-0D98-49E2-B090-00857DCA9D99@gmail.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>	<8209f740710071140n6b4a638alc7bf5e45414c3f72@mail.gmail.com>
	<5DE85FE2-0D98-49E2-B090-00857DCA9D99@gmail.com>
Message-ID: <470955A5.9040001@it.uu.se>

G Bulmer wrote:
> ...I am thinking of staying on for a coupe of days.
> Is this a pleasant part of the city to stay, or would you (or anyone)  
> recommend some other part of the city?

No, you probably want to stay somewhere in the city centre. Precisely
where doesn't matter too much since you can get around quite well on
foot. Most of what you'll want to see is within a radius of no more
than 1-1.5 miles from the Old Town (Gamla Stan). The conference venue,
on the other hand, is about 4 miles out from the city centre, as
the crow flies. It only takes 5-10 minutes to get there on a local
train, or you could take a taxi if you like parting with your money.

    /Richard


From gbulmer@REDACTED  Sun Oct  7 23:56:04 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sun, 7 Oct 2007 23:56:04 +0200
Subject: [erlang-questions] Specifying Type Contracts in Erlang
In-Reply-To: <337538cb0710071241n7cf201b4w78a9aca57d540aa9@mail.gmail.com>
References: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
	<337538cb0710071241n7cf201b4w78a9aca57d540aa9@mail.gmail.com>
Message-ID: <D0D85D77-9AAF-413D-A39B-4EE619FA15F5@gmail.com>

I'm sorry, but only have a paper copy from the proceedings of the  
workshop.
It is marked as copyright ACM, and I'm not sure what that really  
means, i.e. maybe you can get a copy from them.

The paper was presented at the Erlang workshop by Tobias Lindahl.  
Maybe he can help.

I also noticed from the papers references that the publication sited  
was S Marlow, and P Wadler, sorry Simon.

GB


On 7 Oct 2007, at 21:41, Kirill Zaborski wrote:

> Is this paper publicly available somewhere? I think it would be  
> interesting to read.
>
> Best regards,
> Kirill.
>
> On 10/7/07, G Bulmer < gbulmer@REDACTED> wrote:I've been reading  
> the paper "A language for specifying type contracts
> in Erlang and its interaction with success typing' that was presented
> at the Erlang workshop. I think this is a good, useful and valuable
> piece of work.
>
> As far as I can see, there are only 2 pieces which are not covered by
> existing Erlang syntax:
> 1. a range of values, like (0..255), and (0..), and
> 2. "..." which seems to be interpreted as 0 or more.
>
> IMHO, it would be a good idea to stay within Erlangs existing syntax,
> and avoid extensions.
> Case one, could be dealt with using a sequence, seq(0,255) with a
> reserved value for infinity, maybe inf(). I'd be happy inventing a
> new function, it doesn't have to be seq().
> Case two appears to be the same as _. The example was [char(), ...]
> for non-empty string, which could also be written [char()|_]. I'm
> happy to use other alternatives like a 'magic' function, e.g. tail().
>
> What benefits do I think we'd get by staying within Erlang syntax?
> 1. There is little new to learn.
> 2. The parser for type contracts (type annotations) is already
> written. This lowers the bar for others to reuse the type annotation
> information. I believe there is already a well defined representation
> for parsed Erlang which may reduce the amount of work to reuse
> annotations even more.
> 3. I believe there is already a well-defined mechanism for saving
> parsed Erlang in Beam files, so a tool needn't deal with Erlang
> source at all.
> 4. Erlang syntax could be extended in the future, without having any
> syntax 'stolen' by type annotations.
>
> It was pointed out in the workshop that the syntax was proposed by
> Philip Wadler and Simon Peyton Jones. I accept these guys understand
> more about types than I ever will. But, I don't think they are likely
> to confuse surface syntax with underlying semantics. The underlying
> semantics seem to be clear. I am willing to believe I have missed
> some subtlety, but otherwise staying within Erlangs existing syntax
> seems to have the most benefits.
>
> If there is a lot of work which has been expended on building
> annotations with the original annotation syntax, then it would be
> worth building a tool to translate the ... and 1..n syntax. I assume
> it should be relatively straightforward to develop a pretty printer
> based on the existing type contract parser.
>
> So that is my main point, but I started to think, if the type
> contract/annotation syntax is cast within the syntax of normal
> Erlang, then we can consider using it within normal Erlang code. I
> don't think it gives any more power than guards, but it would be a
> pretty unification.
>
> So, we might consider taking a type annotation and using it directly,
> for example in patterns:
> receive X when [char()|tail()], pos_integer() -> fun(X) end.
> Or even:
> Pid ! [char()|tail()], pos_integer().
>
> Also, assuming type-name functions yielded well defined values,
> storing type annotations in ETS/DETS should be straightforward. That
> might be a nice base for nifty tools which would like to check type
> annotations, maybe between versions of code.
>
> Summary: I think here are some genuine benefits for type contracts to
> stay within Erlang syntax.
> G Bulmer
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



From saleyn@REDACTED  Mon Oct  8 03:32:36 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Sun, 07 Oct 2007 20:32:36 -0500
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
 N} socket option?
In-Reply-To: <200710071609.l97G92Gd085783@pluto.hedeland.org>
References: <200710071609.l97G92Gd085783@pluto.hedeland.org>
Message-ID: <470988B4.3050309@gmail.com>

Per Hedeland wrote:
> Serge Aleynikov <saleyn@REDACTED> wrote:
>> gen_tcp:connect/2, gen_tcp:listen/2 functions expose an option {fd, 
>> FileDescriptor} that allows to take an elsewhere opened file descriptor 
>> and make it usable in the gen_tcp module.  This is a very convenient 
>> option, however, what I discovered was that the default options that are 
>> being set on a socket by prim_inet:accept_opts/2 call involved in 
>> gen_tcp:accept/1 try to get/set a 'priority' option that may not be 
>> supported
> 
> Well, it's rather dubious to pass in a non-inet socket to the inet
> modules/driver, 

What else would be the use of the {fd, N} option in gen_tcp:connect/2, 
gen_tcp:listen/2?  I would think that Unix Domain Sockets (UDS) is a 
rather good example or else one would need to mirror all 
gen_tcp/inet_tcp/prim_inet functionality to support UDS.

Ideally the inet_drv along with gen_tcp/inet_tcp/prim_inet would be 
extended to be able to support UDS on Unix-like platforms.

> but anyway the problem you're running into isn't really
> that 'priority' isn't supported (SO_PRIORITY is a Linux-specific thing
> AFAIK, but it's a socket-level option and so works for any type of
> socket), but rather that the driver does some pretty ugly things in an
> attempt to make 'priority' and 'tos' appear to be independent, even
> though they aren't at the OS level (it's a mistake to do this I think,
> but let's ignore that for the moment).

Ah, I suspected that after reading comments in setopt_prio_tos_trick(). ;-)

> This has the effect that it ends up trying to set the IP_TOS option on
> your unix-domain socket, which doesn't work of course, and then it gives
> up on the whole thing even though it was actually SO_PRIORITY that was
> requested (from the prim_inet code you quoted).
> 
> Below is a patch that addresses this - it's against R10B-10 though, and
> won't apply against current versions since even though the code is
> essentially the same there, it has moved around a bit. But maybe you can
> convert it - or just comment out the whole setopt_prio_tos_trick() thing
> there.:-)

Thank you very much, this is very helpful!

> I have subsequently hacked prim_inet and inet_drv to handle arbitrary
> non-inet sockets "properly", i.e. allow for passing in 'unspec' (as in
> AF_UNSPEC) as the address family, and in that case refrain from doing
> *any* AF_INET/AF_INET6-specific get/setsockopts, as well as
> getsockname() and getpeername() (since the address format is unknown).
> It's perhaps still a bit dubious to "abuse" the inet code this way
> though, but you sure get a lot of stuff for free that way...

Perhaps you could send these patches as well and the OTP team could 
consider including them in the distribution?

Regards,

Serge



From ok@REDACTED  Mon Oct  8 03:27:03 2007
From: ok@REDACTED (ok)
Date: Mon, 8 Oct 2007 14:27:03 +1300
Subject: [erlang-questions] Can't build on new Sun T2?
In-Reply-To: <o1abqwejz5.fsf@erix.ericsson.se>
References: <200710052351.l95NpdHD001561@harpo.it.uu.se>
	<o1abqwejz5.fsf@erix.ericsson.se>
Message-ID: <CE75D6A2-9250-4DD8-A06F-1B2C65528E88@cs.otago.ac.nz>

On 6 Oct 2007, at 9:16 pm, Bjorn Gustavsson wrote:
> Good idea. Let's turn off auto-enabling on Solaris/Sparc in R12B.

NO, just try getting it right.  It worked for me.  I don't use gas
*ever*.  In fact it came as something of a surprise that it was even
installed.  Why does HiPE even want gas?

I note that there is a very good reason for a Solaris system
to be using a gcc which doesn't use gas, and that is when it is really
'gccfss' (GCC For Solaris Systems), which uses a GCC front end and a
Sun back end, so that you get the best of both worlds:  GCC language
extensions and Sun optimisation competence at the best price: your
time to install it.  That is, in fact, my current gcc. 
  


From machinshin2002@REDACTED  Mon Oct  8 05:49:02 2007
From: machinshin2002@REDACTED (Vat Raghavan)
Date: Sun, 7 Oct 2007 20:49:02 -0700 (PDT)
Subject: [erlang-questions] Attracting Functional Programmers
Message-ID: <447269.1783.qm@web31502.mail.mud.yahoo.com>

that's not wholly accurate. a very good friend of mine runs a currently small VPS that offers erlang

http://www.guaranteedvps.com/

they offer very good service and competitive prices. 

(wasn't trying to spam or advertise; but hey it came up :) )


--vat


----- Original Message ----
From: G Bulmer <gbulmer@REDACTED>
To: Dale Harvey <harveyd@REDACTED>
Cc: Erlang-Questions (E-mail) <erlang-questions@REDACTED>
Sent: Sunday, October 7, 2007 12:36:23 PM
Subject: Re: [erlang-questions] Attracting Functional Programmers


On 7 Oct 2007, at 17:37, Dale Harvey wrote:

> In terms of attracting programmers, it seems like my main reason,  
> if noticing the traction that erlang has been gaining recently, to  
> not decided to give erlang a shot is the fact that I would have to  
> own my own server in order to develop anything public in it.
>
> Personally I have tested quite a few web based languages, all of  
> which were little experiments which I could 'deploy' on my shared  
> hosting. If it werent for other circumstances I almost certainly  
> wouldnt have tested erlang.

I'm not sure I understand your point.
Are you saying Erlang/OTP will become much more useful to people like  
you when it is made available as a "shrink wrapped" platform by  
hosting companies?
If that's your point, I agree.

AFAIK, the only company offering (or planing to offer) hosted Erlang  
was joyent, and Joyant were expensive compared to the majority of  
hosting companies.
As a comparison, Railsplayground.com offers Ruby on Rails hosting  
starting at $5/month. You just upload your app., and go.

The cheapest way for a company to offer Erlang hosting would be  
within virtual private servers (VPS), e.g. Linux Virtual Machines  
running Erlang Nodes. I've googled around and can't find anyone doing  
that. Of course, you could build the virtual private server yourself,  
but that seems like significant effort which a hosting company could  
just 'solve' for people.
Is there a pre-built VPS image to take out all of the pain?

A nice solution would be to have a "pre-configured, templated image"  
of an Erlang environment and Erlang/OTP at Amazon Elastic Compute  
Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011
Amazon's computing cloud supports starting up and closing down upnew  
servers on-demand, you only pay for what you use. This would be very  
handy if your throughput requirements are very peaky.

GB

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions





       
____________________________________________________________________________________
Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. 
http://mobile.yahoo.com/go?refer=1GNXIC


From bjorn@REDACTED  Mon Oct  8 07:43:50 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 08 Oct 2007 07:43:50 +0200
Subject: [erlang-questions] Code vs. binary heap storage
In-Reply-To: <B73FAC3A-D103-479B-A430-209ACC1659BE@duomark.com>
References: <B73FAC3A-D103-479B-A430-209ACC1659BE@duomark.com>
Message-ID: <o1hcl2notl.fsf@erix.ericsson.se>

Jay Nelson <jay@REDACTED> writes:

> Given the following code:
> 
> return_bin(pinocchio) ->  <<"Once upon a time... ">>;
> return_bin(cinderella) -> <<"In a castle long ago...">>;
> return_bin(star_wars) -> <<"In a galaxy far, far way...">>.
> 
> Assume each of the binaries is of some significant size > 32 bytes.
>
> If the module containing this function is code-loaded into a node, I  
> want to verify whether the following are true:
> 
> 1) The binary only exists once regardless of how many processes or  
> modules access the functions.
> 2) The binary for each branch of the function is stored in the binary  
> heap.
> 3) The function return is a pointer to the existing binary heap element.

No to all three questions. A new binary is constructed every time the
code is called, regardless of which process calls it.

> 4) The memory footprint of a process which can call these functions  
> does not include the size of the binaries, even if the functions are  
> called and a process variable is bound to the value.

Yes, the binary itself will be stored outside of the process (provided
the size of the binary is > 64).

> 5) These binaries can never be garbage collected unless the module  
> containing the functions is unloaded (and all other references  
> obtained by calling the functions are released).

No. None of the binaries reference the loaded code.

> 6) Having a process per binary and a process to route requests would  
> not be more or less memory efficient than having a single process  
> with the above function in place of the routing process.

One or or more processes doesn't matter.


To share the binaries, you should call return_bin/1 only once for each
binary and store the result in some sort of dictionary (dict, gb_trees,
or process dictionary) or ets table, then retrieve the binary from there
when you need it.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From mikpe@REDACTED  Mon Oct  8 07:47:31 2007
From: mikpe@REDACTED (Mikael Pettersson)
Date: Mon, 8 Oct 2007 07:47:31 +0200 (MEST)
Subject: [erlang-questions] Can't build on new Sun T2?
Message-ID: <200710080547.l985lVif015154@harpo.it.uu.se>

On Mon, 8 Oct 2007 14:27:03 +1300, ok <ok@REDACTED> wrote:
> On 6 Oct 2007, at 9:16 pm, Bjorn Gustavsson wrote:
> > Good idea. Let's turn off auto-enabling on Solaris/Sparc in R12B.
> 
> NO, just try getting it right.  It worked for me.  I don't use gas
> *ever*.  In fact it came as something of a surprise that it was even
> installed.  Why does HiPE even want gas?
> 
> I note that there is a very good reason for a Solaris system
> to be using a gcc which doesn't use gas, and that is when it is really
> 'gccfss' (GCC For Solaris Systems), which uses a GCC front end and a
> Sun back end, so that you get the best of both worlds:  GCC language
> extensions and Sun optimisation competence at the best price: your
> time to install it.  That is, in fact, my current gcc. 

On Solaris/SPARC, HiPE doesn't "want" gas. It just so happens that
many users have a pre-built gcc (supplied by Sun I presume) that's
been configured to use GNU as with Sun's ld, and that combo breaks
for unknown reasons on our code.

You don't see the problem because you have a different gcc, and the HiPE
group's gcc has been configured without --with-gnu-as to avoid the problem.

The problem will be debugged/fixed/worked-around as part of HiPE's SPARC
code rewrite scheduled to start in a couple of weeks. One way or another,
R12B won't have the problem.

/Mikael


From per@REDACTED  Mon Oct  8 08:12:34 2007
From: per@REDACTED (Per Hedeland)
Date: Mon, 8 Oct 2007 08:12:34 +0200 (CEST)
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
	N} socket option?
In-Reply-To: <470988B4.3050309@gmail.com>
Message-ID: <200710080612.l986CYqT003566@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>Per Hedeland wrote:
>> Serge Aleynikov <saleyn@REDACTED> wrote:
>>> gen_tcp:connect/2, gen_tcp:listen/2 functions expose an option {fd, 
>>> FileDescriptor} that allows to take an elsewhere opened file descriptor 
>>> and make it usable in the gen_tcp module.  This is a very convenient 
>>> option, however, what I discovered was that the default options that are 
>>> being set on a socket by prim_inet:accept_opts/2 call involved in 
>>> gen_tcp:accept/1 try to get/set a 'priority' option that may not be 
>>> supported
>> 
>> Well, it's rather dubious to pass in a non-inet socket to the inet
>> modules/driver, 
>
>What else would be the use of the {fd, N} option in gen_tcp:connect/2, 
>gen_tcp:listen/2?

Primarily the case where you need to bind a socket to a "privileged"
port, but don't want to run the whole Erlang system as root. You can
then (create and) bind the socket in an external process, and pass it to
Erlang via sendmsg(2) and a driver. You could also have "inherited" the
socket descriptor somehow, e.g. if you start Erlang via [x]inetd,
stdin/out/err is a socket - this can also be a way to handle the
privileged port case: Bind the socket in a setuid-root or root-started
program that then setuid()s to an unprivileged uid and execs the Erlang
runtime system.

>  I would think that Unix Domain Sockets (UDS) is a 
>rather good example or else one would need to mirror all 
>gen_tcp/inet_tcp/prim_inet functionality to support UDS.
>
>Ideally the inet_drv along with gen_tcp/inet_tcp/prim_inet would be 
>extended to be able to support UDS on Unix-like platforms.

Yeah, it's nice to get all that stuff for "UDS", but I think it's clear
from the code that this was not the intention. The inet driver clearly
assumes AF_INET[6] sockets, not just by way of its name. And UDS doesn't
do TCP (or UDP) - of course the difference from generic SOCK_STREAM or
SOCK_DGRAM is pretty minimal (mainly the set of available options), but
gen_tcp and gen_udp would at least be "inappropriate" names...

>> Below is a patch that addresses this - it's against R10B-10 though, and
>> won't apply against current versions since even though the code is
>> essentially the same there, it has moved around a bit. But maybe you can
>> convert it - or just comment out the whole setopt_prio_tos_trick() thing
>> there.:-)
>
>Thank you very much, this is very helpful!

I converted the patch to match R11B-5, it was pretty trivial, but let me
know if you want it (untested though).

>> I have subsequently hacked prim_inet and inet_drv to handle arbitrary
>> non-inet sockets "properly", i.e. allow for passing in 'unspec' (as in
>> AF_UNSPEC) as the address family, and in that case refrain from doing
>> *any* AF_INET/AF_INET6-specific get/setsockopts, as well as
>> getsockname() and getpeername() (since the address format is unknown).
>> It's perhaps still a bit dubious to "abuse" the inet code this way
>> though, but you sure get a lot of stuff for free that way...
>
>Perhaps you could send these patches as well and the OTP team could 
>consider including them in the distribution?

If they want them, sure (actually the EPL says that I must:-), but I'm
not sure that they do... In any case you don't really need them for UDS,
the inet driver is very "tolerant" of failing get/setsockopts (except in
the case of SO_PRIORITY/IP_TOS:-), so it "just works".

I had a broader requirement though, to support any SOCK_STREAM type,
allowing the customer to plug in the needed interface code, and didn't
want to have to answer questions about why we were attempting to do
INET-specific things with it.:-) More seriously, there is a very real
possibility that getsockname()/getpeername() calls assuming AF_INET[6]
will fail in the case of an address family that is neither INET nor
UNIX/LOCAL - the inet driver does these w/o being asked in some cases
(in particular when passing in an existing file descriptor:-), and
failure there is fatal.

Note also that I don't use gen_tcp for passing in the descriptor and so
haven't modified it for the 'unspec' family - a mostly-wrapper module
with the same interface as gen_tcp talks to the driver that creates the
socket, and passes the descriptor in via inet:fdopen/5 - maybe
marginally "cleaner", but of course it just turns around and calls
gen_tcp for accept/recv/send etc, and you still get 'tcp' in the tuples
received in active mode.

--Per


From ulf.wiger@REDACTED  Mon Oct  8 10:24:20 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Mon, 08 Oct 2007 10:24:20 +0200
Subject: [erlang-questions] Trying to use gen_leader and gdict
In-Reply-To: <855EC6C7-0EBD-4923-878E-34FED162FE5A@geophile.com>
References: <E0A0DD86-5087-4349-B3A6-40219627A949@geophile.com>
	<4701F5E5.2080109@ericsson.com>
	<855EC6C7-0EBD-4923-878E-34FED162FE5A@geophile.com>
Message-ID: <4709E934.3020007@ericsson.com>


Try the following:

1) You should identify nodes as _either_ candidates _or_ workers,
    not both at the same time. I'm not sure what the actual effect
    will be (I think they will count as candidates)

2) Make sure that a gdict instance is also started on at least
    one of the candidate nodes.

3) The candidates must be aware of the nodes intending to function
    as workers.

In your test, it appears as if you run the test on z@REDACTED, but
this node is neither a candidate nor a worker, at least in the
command you issue on z@REDACTED It's not clear whether any
gen_leader instances are running on a@REDACTED or b@REDACTED, but even
if there are, z@REDACTED will not be included if it's not listed
either as a worker or candidate on a@REDACTED or b@REDACTED

There is currently no facility in gen_leader to add candidates
or workers dynamically. It can be done during a code change.
More dynamic behaviour will hopefully be added eventually.

BR,
Ulf W


Jack Orenstein wrote:
> On Oct 2, 2007, at 3:40 AM, Ulf Wiger (TN/EAB) wrote:
> 
>> Jack Orenstein wrote:
>>> I am running Erlang R11B on OS X and have been investigating
>>> gen_leader.  I'm having some trouble with the gdict example included
>>> with gen_leader. I wrote the following program to use gdict:
>>> ...
>>> This causes a timeout:
>>>      zack$ time erl -sname z -noshell -s test main -s init stop
>>>      init({dict,0,
>>>                 16,
>>>                 16,
>>>                 8,
>>>                 80,
>>>                 48,
>>>                 {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
>>>                 {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}})
>>>      {"init terminating in do_boot",{timeout,{gen_leader,leader_call, 
>>> [<0.34.0>,{store,#Fun<gdict.0.10659195>}]}}}
>>> ...
>>> Can anyone see what I'm doing wrong?
>>
>> You're not doing anything wrong. There's a bug in gen_leader.
>> When you start gen_leader with only one candidate node, it
>> will still go into a receive loop waiting for responses from
>> other candidates.
> 
> Thanks for your response. I'm seeing the same problem with three nodes 
> (one running the test, two candidates/workers).
> 
> Here is my modified test:
> 
> -module(test).
> -export([main/0]).
> 
> -define(DUMP(X), io:format("~p:~p - ~p = ~p~n", [?MODULE, ?LINE, ??X, X])).
> 
> main() ->
>     rpc:call(a@REDACTED, hello, hello, []),
>     rpc:call(b@REDACTED, hello, hello, []),
>     Nodes = nodes(),
>     {ok, D} = gdict:new(test, Nodes, Nodes),
>     ?DUMP(D),
>     ?DUMP(node()),
>     ?DUMP(Nodes),
>     ?DUMP(gdict:append(a, 1, D)),
>     ?DUMP(gdict:append(b, 2, D)),
>     ?DUMP(gdict:append(c, 3, D)),
>     ?DUMP(gdict:find(a, D)),
>     ?DUMP(gdict:find(b, D)),
>     ?DUMP(gdict:find(c, D)).
> 
> I started two nodes (a@REDACTED, b@REDACTED) and ran the test. Here is the 
> output, with timing (I added some ?DUMP calls to gdict and gen_leader too):
> 
> gdict:42 - "Name" = test
> gdict:43 - "Candidates" = [a@REDACTED,b@REDACTED]
> gdict:44 - "Workers" = [a@REDACTED,b@REDACTED]
> init({dict,0,
>            16,
>            16,
>            8,
>            80,
>            48,
>            {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
>            {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}})
> gdict:46 - "GenLeaderStart" = {ok,<0.42.0>}
> test:11 - "D" = <0.42.0>
> test:12 - "node ( )" = z@REDACTED
> test:13 - "Nodes" = [a@REDACTED,b@REDACTED]
> gdict:61 - "Key" = a
> gdict:62 - "Value" = 1
> gen_leader:259 - "Name" = <0.42.0>
> gen_leader:260 - "Request" = {store,#Fun<gdict.0.10659195>}
> {"init terminating in 
> do_boot",{timeout,{gen_leader,leader_call,[<0.42.0>,{store,#Fun<gdict.0.10659195>}]}}} 
> 
> 
> Crash dump was written to: erl_crash.dump
> init terminating in do_boot ()
> 
> real    0m5.986s
> user    0m0.199s
> sys    0m0.389s
> 
> 
> 
> Jack



From yinso.chen@REDACTED  Mon Oct  8 10:30:25 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 01:30:25 -0700
Subject: [erlang-questions] gen_server vs gen_fsm
Message-ID: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>

Hi all -

I have a few newbie questions regarding gen_server and gen_fsm - I couldn't
find the answers easily though I'm sure they exists somewhere; would
appreciate pointers and sharing of experiences in designs :)

* are gen_fsm also a "server"?  It seems like gen_fsm is used to manage a
process that have multiple stages to fulfill its job, where gen_server by
and large has only one state, is that correct?  If not, what are the
differences?

* when should one use gen_fsm?  and when should one use a gen_server?

* I've read that the best way to handle multiple requests is to use
{noreply, State} in conjunction to spawn worker processes to handle the
request - should the worker process also be gen_server or gen_fsm?  For
example, what should a web server's worker process be?

* what's a good way to test the spawning to make it that it works?  I mean,
are there available tools to simplify the process?

Thanks very much,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/342638a3/attachment.htm>

From yinso.chen@REDACTED  Mon Oct  8 10:39:32 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 01:39:32 -0700
Subject: [erlang-questions] ODBC parameters: unsupported types
Message-ID: <779bf2730710080139n232d3643u731b0fec9d53eb4c@mail.gmail.com>

Hi all -

Based on http://www.erlang.org/doc/apps/odbc/databases.html#3.3, types such
as datetime and blob are currently unsupported.  Are there plans to support
these in the future?  How do people currently enter data with these types,
besides manually concatenate sql string?

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/834ad6de/attachment.htm>

From chandrashekhar.mullaparthi@REDACTED  Mon Oct  8 10:52:58 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Mon, 8 Oct 2007 09:52:58 +0100
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
Message-ID: <b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>

Hi,

On 08/10/2007, YC <yinso.chen@REDACTED> wrote:
> Hi all -
>
> I have a few newbie questions regarding gen_server and gen_fsm - I couldn't
> find the answers easily though I'm sure they exists somewhere; would
> appreciate pointers and sharing of experiences in designs :)
>
> * are gen_fsm also a "server"?  It seems like gen_fsm is used to manage a
> process that have multiple stages to fulfill its job, where gen_server by
> and large has only one state, is that correct?  If not, what are the
> differences?

A gen_fsm helps you implement the logic for a finite state machine.
You can implement a FSM using gen_server but gen_fsm makes it much
more easier and your code will be more structured and easier to read.

>
> * when should one use gen_fsm?  and when should one use a gen_server?

Use a gen_server when you want a long lived process which reacts to
events. Use a gen_fsm to implement state machines. See
http://en.wikipedia.org/wiki/Finite_state_machine

> * I've read that the best way to handle multiple requests is to use
> {noreply, State} in conjunction to spawn worker processes to handle the
> request - should the worker process also be gen_server or gen_fsm?  For
> example, what should a web server's worker process be?

A web server's worker process doesn't have to be a gen_server or a
gen_fsm. It just needs to be a module which given the HTTP request,
does some processing and returns some content to the HTTP client.

> * what's a good way to test the spawning to make it that it works?  I mean,
> are there available tools to simplify the process?

Sorry - I don't understand your question.

cheers
Chandru


From davidnwelton@REDACTED  Mon Oct  8 11:03:42 2007
From: davidnwelton@REDACTED (David Welton)
Date: Mon, 8 Oct 2007 11:03:42 +0200
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <9b08084c0710070306j457cb5a7hbb5ad2a934d16098@mail.gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
	<9b08084c0710070306j457cb5a7hbb5ad2a934d16098@mail.gmail.com>
Message-ID: <9877cd600710080203m63a1e2a3h2478a61c3247ea24@mail.gmail.com>

> I think we have to answer the question:
>
>     "what is the compelling reason to use Erlang?"

Right.  And it really has got to be something that it does *much*
better.  A 10% advantage won't make people give up their
libraries/acquired knowledge in other languages/etc...

> Its not ('cos it's FP)

> The  2 second pitch which I used at JAOO was
>
> Erlang = Parallel programming without pain
>
> I'm not sure if this is best - it could be
>
> Fault-tolerant ptogramming without pain or
>
> Multicore programming wthout pain

>From my point of view, "parallel programming" is more immediate than
"fault tolerant".  It's simple, and very clear that concurrency in
most popular languages is painful.  "Fault tolerant" probably has less
appeal to the average programmer who muddles through with whatever
they use currently.  I think it's a vaguer image to sell, as well.
Sounds like an added bonus rather than the key selling point.  There
are probably people who would regard it as key, but there are likely
more who are interested in concurrency.

Thanks,
-- 
David N. Welton
http://www.welton.it/davidw/


From gordonguthrie@REDACTED  Mon Oct  8 11:23:53 2007
From: gordonguthrie@REDACTED (Gordon Guthrie)
Date: Mon, 8 Oct 2007 10:23:53 +0100 (BST)
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <890BC292-0A17-40A7-A9E6-6400FECCA1C2@gmail.com>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com><9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com><BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com><fc6cbe440710070837l1c49c6b9y445df9a64a9ee5b7@mail.gmail.com>
	<890BC292-0A17-40A7-A9E6-6400FECCA1C2@gmail.com>
Message-ID: <49557.77.99.143.215.1191835433.squirrel@backawinner.gg>

Gary

> A nice solution would be to have a "pre-configured, templated image"
> of an Erlang environment and Erlang/OTP at Amazon Elastic Compute
> Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011

Dale and I are working on a single-click install Erlang developers
platform based on the Ubuntu liveCD that comes preloaded with development
tools...

This uses the Ubuntu Customization Kit to layer a small slice o'Erlang
over Dapper Drake.

There are a couple of problems with Erlang/Mnesia under the ECC - the most
noticable one is how you get your data backed off. We are also working on
an  Mnesia cluster management console that would allow you to set up 2 or
three ECC instances and distribute the data - and then do a table dump off
the Amazon storage cloud on a daily basis...

*Eventually* this stuff will be open sourced...

Gordon

>
> On 7 Oct 2007, at 17:37, Dale Harvey wrote:
>
>> In terms of attracting programmers, it seems like my main reason,
>> if noticing the traction that erlang has been gaining recently, to
>> not decided to give erlang a shot is the fact that I would have to
>> own my own server in order to develop anything public in it.
>>
>> Personally I have tested quite a few web based languages, all of
>> which were little experiments which I could 'deploy' on my shared
>> hosting. If it werent for other circumstances I almost certainly
>> wouldnt have tested erlang.
>
> I'm not sure I understand your point.
> Are you saying Erlang/OTP will become much more useful to people like
> you when it is made available as a "shrink wrapped" platform by
> hosting companies?
> If that's your point, I agree.
>
> AFAIK, the only company offering (or planing to offer) hosted Erlang
> was joyent, and Joyant were expensive compared to the majority of
> hosting companies.
> As a comparison, Railsplayground.com offers Ruby on Rails hosting
> starting at $5/month. You just upload your app., and go.
>
> The cheapest way for a company to offer Erlang hosting would be
> within virtual private servers (VPS), e.g. Linux Virtual Machines
> running Erlang Nodes. I've googled around and can't find anyone doing
> that. Of course, you could build the virtual private server yourself,
> but that seems like significant effort which a hosting company could
> just 'solve' for people.
> Is there a pre-built VPS image to take out all of the pain?
>
> A nice solution would be to have a "pre-configured, templated image"
> of an Erlang environment and Erlang/OTP at Amazon Elastic Compute
> Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011
> Amazon's computing cloud supports starting up and closing down upnew
> servers on-demand, you only pay for what you use. This would be very
> handy if your throughput requirements are very peaky.
>
> GB
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



From yinso.chen@REDACTED  Mon Oct  8 11:13:07 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 02:13:07 -0700
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
Message-ID: <779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>

Hi -

Thanks for the quick response, please see inline.

On 10/8/07, Chandru <chandrashekhar.mullaparthi@REDACTED> wrote:

> > A gen_fsm helps you implement the logic for a finite state machine.
> > You can implement a FSM using gen_server but gen_fsm makes it much
> > more easier and your code will be more structured and easier to read.
>
> > > * when should one use gen_fsm?  and when should one use a gen_server?
>
> > Use a gen_server when you want a long lived process which reacts to
> > events. Use a gen_fsm to implement state machines. See
> > http://en.wikipedia.org/wiki/Finite_state_machine


When I think of state machine two primary types of example comes to mind:

1) the machine will generate the end result in one call as the input stream
contains enough information to allow the machine to finish its processing;
for example, a regular expression match

2) the machine will reach its end state only after multiple calls as each
call changes the state; for example, an e-commerce website.

Which type is gen_fsm suited for?  To me, the 1st type is not a server,
where the second type is, and hence my confusion about its suitability.  As
you say that gen_server is better suited for long lived processes, is
gen_fsm not suited to implement the logic behind an e-commerce site?

> A web server's worker process doesn't have to be a gen_server or a
> > gen_fsm. It just needs to be a module which given the HTTP request,
> > does some processing and returns some content to the HTTP client.


I see.  Are there disadvantages to use gen_server or gen_fsm in this case?
Overkill?

> > * what's a good way to test the spawning to make it that it works?  I
> mean,
> > > are there available tools to simplify the process?
>
> > Sorry - I don't understand your question.


Sorry - I simply meant to ask if there are tools within Erlang that would
help with simulating load testings, such as the ab tool.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/c9baabe3/attachment.htm>

From tobias.lindahl@REDACTED  Mon Oct  8 12:02:41 2007
From: tobias.lindahl@REDACTED (Tobias Lindahl)
Date: Mon, 08 Oct 2007 12:02:41 +0200
Subject: [erlang-questions] Specifying Type Contracts in Erlang
In-Reply-To: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
References: <1A09638C-1AE8-4E22-8237-C40F1CC03E1C@gmail.com>
Message-ID: <470A0041.6040607@it.uu.se>

EDIT: After having written most of my answers in this mail I want to 
make a general point. The types would be parsed by the Erlang parser, 
and not some plug-in parser that understands types. The types will be 
turned into parse forms that are Erlang terms, and then be used by any 
Erlang tool. As long as you use the Erlang parser (which I guess most 
tools are using) there is only the matter of defining how the parse 
forms look like, and to provide an interface for interpreting them. I 
think it must be clear from the context if you are talking about types 
or erlang terms, so I think there should be a special parse form for types.

G Bulmer wrote:
> I've been reading the paper "A language for specifying type contracts  
> in Erlang and its interaction with success typing' that was presented  
> at the Erlang workshop. I think this is a good, useful and valuable  
> piece of work.

Thank you. However, if nobody uses it it will be worthless, so let us 
try to find a syntax that pleases most erlangers.

It was pointed out to me that there are differences between our syntax 
and the one used by Quick Check and possibly also with Edoc. I think it 
is ok to have differences as long as the core of the language is the 
same so it would be good if we can agree on some syntax, and pretty fast 
at that since we would like to include the type specifications in R12B.

> As far as I can see, there are only 2 pieces which are not covered by  
> existing Erlang syntax:
> 1. a range of values, like (0..255), and (0..), and
> 2. "..." which seems to be interpreted as 0 or more.
> 
> IMHO, it would be a good idea to stay within Erlangs existing syntax,  
> and avoid extensions.
> Case one, could be dealt with using a sequence, seq(0,255) with a  
> reserved value for infinity, maybe inf(). I'd be happy inventing a  
> new function, it doesn't have to be seq().

We used to have ranges with the syntax range(1,42) which is more or less 
what you suggest. I think that range is a slightly better keyword since 
seq somewhat suggests that you really have a sequence including all 
integers (which of course is more or less the same as range...).

> Case two appears to be the same as _. The example was [char(), ...]  
> for non-empty string, which could also be written [char()|_]. I'm  
> happy to use other alternatives like a 'magic' function, e.g. tail().

The "proper" description for this type is nonempty_list(char()), but as 
this is a pretty common construction it is good to have syntactic sugar 
for it. [char()|_] looks a bit to me as a list with a char as the first 
element and then anything in the tail, which is not the meaning of the 
type. The meaning of the type is better captured by [char()|[char()]], 
but this is also a bit of a mouthful. The current representation 
[char(),...] is quite handy in that respect.

There two more places where we do not have erlang syntax, and that is 
the function types. Both as specifications, and as higher order 
functions (funs). The specifications are a minor problem since it can 
only appear at the top level of a -spec, but the funs are a bit more 
problematic. In our current implementation they look as

fun((T1, T2, T3)-> T4)

and if we would use erlang fun syntax it would look as

fun(T1, T2, T3) -> T4 end

whith the end keyword having to be added to end the type.

> What benefits do I think we'd get by staying within Erlang syntax?
> 1. There is little new to learn.
You still need to understand that you are talking about types and not 
about function calls and structured term construction, so I don't think 
this is a major point.

> 2. The parser for type contracts (type annotations) is already  
> written. This lowers the bar for others to reuse the type annotation  
> information. I believe there is already a well defined representation  
> for parsed Erlang which may reduce the amount of work to reuse  
> annotations even more.

The parse format for types should probably recognize that the user 
specified the type byte() rather than a call to the function byte(), but 
this is of course an implementation issue.

> 3. I believe there is already a well-defined mechanism for saving  
> parsed Erlang in Beam files, so a tool needn't deal with Erlang  
> source at all.

The mechanism is the same whatever the parser gives back.

> 4. Erlang syntax could be extended in the future, without having any  
> syntax 'stolen' by type annotations.

Yes, this is a good point. We would still probably hog the '::' to 
specify that you are talking about types rather than of Erlang terms.

> If there is a lot of work which has been expended on building  
> annotations with the original annotation syntax, then it would be  
> worth building a tool to translate the ... and 1..n syntax. I assume  
> it should be relatively straightforward to develop a pretty printer  
> based on the existing type contract parser.

Yes, this is not a major problem.

> So that is my main point, but I started to think, if the type  
> contract/annotation syntax is cast within the syntax of normal  
> Erlang, then we can consider using it within normal Erlang code. I  
> don't think it gives any more power than guards, but it would be a  
> pretty unification.
> 
> So, we might consider taking a type annotation and using it directly,  
> for example in patterns:
> receive X when [char()|tail()], pos_integer() -> fun(X) end.
> Or even:
> Pid ! [char()|tail()], pos_integer().

I will leave for others to debate how types can be used within normal 
Erlang constructs, but it must be absolutely clear from the context that 
the programmer is talking about types, since this would be a major shift 
in the language. Having types and specifications given as attributes is 
a minor shift since we are not altering the semantics of Erlang (unless 
you specifically ask for it by using some tool).

> Also, assuming type-name functions yielded well defined values,  
> storing type annotations in ETS/DETS should be straightforward. That  
> might be a nice base for nifty tools which would like to check type  
> annotations, maybe between versions of code.

The types are of course represented as Erlang terms in my 
implementation, and storing them in whatever form you like is 
unproblematic. However, the types should be treated as an ADT and should 
be handled by a module with a nice interface. This is somewhat 
orthogonal to types having the same syntax as the rest of Erlang.

> Summary: I think here are some genuine benefits for type contracts to  
> stay within Erlang syntax.
> G Bulmer

I can also see benefits with having types in Erlang syntax, but the 
syntax still have to be extended to make it clear from the context that 
the programmer is talking about types and not general Erlang terms. Any 
tool that wants to use types would do so either directly on the parse 
forms or on some abstract format that will be provided. Either way, the 
types would have an Erlang representation that can be manipulated.

Tobias




From chandrashekhar.mullaparthi@REDACTED  Mon Oct  8 12:31:20 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Mon, 8 Oct 2007 11:31:20 +0100
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
Message-ID: <b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>

Hi,

On 08/10/2007, YC <yinso.chen@REDACTED> wrote:
> Hi -
>
> Thanks for the quick response, please see inline.
>
> On 10/8/07, Chandru <chandrashekhar.mullaparthi@REDACTED >
> wrote:
> > > A gen_fsm helps you implement the logic for a finite state machine.
> > > You can implement a FSM using gen_server but gen_fsm makes it much
> > > more easier and your code will be more structured and easier to read.
> >
> > > > * when should one use gen_fsm?  and when should one use a gen_server?
> >
> > > Use a gen_server when you want a long lived process which reacts to
> > > events. Use a gen_fsm to implement state machines. See
> > > http://en.wikipedia.org/wiki/Finite_state_machine
>
> When I think of state machine two primary types of example comes to mind:
>
> 1) the machine will generate the end result in one call as the input stream
> contains enough information to allow the machine to finish its processing;
> for example, a regular expression match
>
> 2) the machine will reach its end state only after multiple calls as each
> call changes the state; for example, an e-commerce website.
>
> Which type is gen_fsm suited for?  To me, the 1st type is not a server,
> where the second type is, and hence my confusion about its suitability.  As
> you say that gen_server is better suited for long lived processes, is
> gen_fsm not suited to implement the logic behind an e-commerce site?
>

gen_fsm can certainly be used to implement the logic behind an
e-commerce site where you'd have one instance of the gen_fsm for each
customer you are handling. You'd somehow have to map (Cookie to
process-id) your HTTP request to a particular instance of the state
machine. If you wanted to keep the state machine persistent, you'd
have to save it to some database and resurrect it when your next
request comes in and your state machine isn't around for whatever
reason.

> > > A web server's worker process doesn't have to be a gen_server or a
> > > gen_fsm. It just needs to be a module which given the HTTP request,
> > > does some processing and returns some content to the HTTP client.
>
> I see.  Are there disadvantages to use gen_server or gen_fsm in this case?
> Overkill?

If it is a HTTP request where the processing of one request is not
related to the next, it is definitely overkill to use a
gen_server/gen_fsm. But if you are talking about a ecommerce site, a
gen_fsm makes sense. But you'd have to take care to store the FSM
state in some persistent database, so that the user can be served even
if  your webserver suffered a restart in between requests.

>
> > > > * what's a good way to test the spawning to make it that it works?  I
> mean,
> > > > are there available tools to simplify the process?
> >
> > > Sorry - I don't understand your question.
>
>
> Sorry - I simply meant to ask if there are tools within Erlang that would
> help with simulating load testings, such as the ab tool.
>

I see. There is Tsung (previously tsunami) though I've never used it.
http://tsung.erlang-projects.org/

cheers
Chandru


From opendev@REDACTED  Mon Oct  8 13:06:13 2007
From: opendev@REDACTED (Joern)
Date: Mon, 8 Oct 2007 13:06:13 +0200
Subject: [erlang-questions] gen_tcp custom framing
In-Reply-To: <47064E36.708@lundata.se>
References: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>
	<47064E36.708@lundata.se>
Message-ID: <9e009ad0710080406k5e0ba800q8e9721dc1f68896a@mail.gmail.com>

Hi Peter,

On 10/5/07, Peter Lund <erlang@REDACTED> wrote:

> Is this UCP or CIMD perhaps? :)

It's UCP, wonderful little gem, really.

> Standard practice or not, I do not know, but some number of times, I
> have now implemented myself buffering of incoming IP-segments into a
> binary and each time new data comes in I try to find one or more complete
> packages in this buffer. If I find it I handle the packages.

Yes. I think I'll just split my whole packet matching code in two
functions, for head and tail matching. The head is fixed size, so that
should work ok.

> In R9, I have also forced to match each packet more than one time since
> an initial length value read out first, cannot be used later in the same
> binary match statement, but I am guessing that this problem now is history
> when using R11.

Hmm - I'm not sure I follow you on this one. Can you elaborate?

Best regards,


Joern
--


From saleyn@REDACTED  Mon Oct  8 14:08:48 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Mon, 08 Oct 2007 07:08:48 -0500
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
 N} socket option?
In-Reply-To: <200710080612.l986CYqT003566@pluto.hedeland.org>
References: <200710080612.l986CYqT003566@pluto.hedeland.org>
Message-ID: <470A1DD0.1030703@gmail.com>

Per Hedeland wrote:
> Primarily the case where you need to bind a socket to a "privileged"
> port, but don't want to run the whole Erlang system as root. You can
> then (create and) bind the socket in an external process, and pass it to
> Erlang via sendmsg(2) and a driver. You could also have "inherited" the
> socket descriptor somehow, e.g. if you start Erlang via [x]inetd,
> stdin/out/err is a socket - this can also be a way to handle the
> privileged port case: Bind the socket in a setuid-root or root-started
> program that then setuid()s to an unprivileged uid and execs the Erlang
> runtime system.

Indeed.  For the setuid requirement I've been using Linux capability 
flags, but I do recall our former discussion and your rightful concern 
of portability issues with this approach.

> I converted the patch to match R11B-5, it was pretty trivial, but let me
> know if you want it (untested though).

Yes, please. Since I haven't applied it yet, I'd rather start with what 
you have for R11B-5 to begin with.

Regards,

Serge



From qrilka@REDACTED  Mon Oct  8 13:11:27 2007
From: qrilka@REDACTED (Kirill Zaborski)
Date: Mon, 8 Oct 2007 15:11:27 +0400
Subject: [erlang-questions] Attracting Functional Programmers
In-Reply-To: <49557.77.99.143.215.1191835433.squirrel@backawinner.gg>
References: <E28A16DA-712F-45C7-93EE-CDDD94BE919E@gmail.com>
	<9877cd600710070137o57939b68l325585bf72d4932c@mail.gmail.com>
	<BF49DCC2-B6A8-4985-9281-DD0270284FCA@gmail.com>
	<fc6cbe440710070837l1c49c6b9y445df9a64a9ee5b7@mail.gmail.com>
	<890BC292-0A17-40A7-A9E6-6400FECCA1C2@gmail.com>
	<49557.77.99.143.215.1191835433.squirrel@backawinner.gg>
Message-ID: <337538cb0710080411i1be2f3f9id96cbb212b9d5786@mail.gmail.com>

That would be quite interesting at least to try out.

Best regards,
Kirill.

On 10/8/07, Gordon Guthrie <gordonguthrie@REDACTED> wrote:
>
> Gary
>
> > A nice solution would be to have a "pre-configured, templated image"
> > of an Erlang environment and Erlang/OTP at Amazon Elastic Compute
> > Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011
>
> Dale and I are working on a single-click install Erlang developers
> platform based on the Ubuntu liveCD that comes preloaded with development
> tools...
>
> This uses the Ubuntu Customization Kit to layer a small slice o'Erlang
> over Dapper Drake.
>
> There are a couple of problems with Erlang/Mnesia under the ECC - the most
> noticable one is how you get your data backed off. We are also working on
> an  Mnesia cluster management console that would allow you to set up 2 or
> three ECC instances and distribute the data - and then do a table dump off
> the Amazon storage cloud on a daily basis...
>
> *Eventually* this stuff will be open sourced...
>
> Gordon
>
> >
> > On 7 Oct 2007, at 17:37, Dale Harvey wrote:
> >
> >> In terms of attracting programmers, it seems like my main reason,
> >> if noticing the traction that erlang has been gaining recently, to
> >> not decided to give erlang a shot is the fact that I would have to
> >> own my own server in order to develop anything public in it.
> >>
> >> Personally I have tested quite a few web based languages, all of
> >> which were little experiments which I could 'deploy' on my shared
> >> hosting. If it werent for other circumstances I almost certainly
> >> wouldnt have tested erlang.
> >
> > I'm not sure I understand your point.
> > Are you saying Erlang/OTP will become much more useful to people like
> > you when it is made available as a "shrink wrapped" platform by
> > hosting companies?
> > If that's your point, I agree.
> >
> > AFAIK, the only company offering (or planing to offer) hosted Erlang
> > was joyent, and Joyant were expensive compared to the majority of
> > hosting companies.
> > As a comparison, Railsplayground.com offers Ruby on Rails hosting
> > starting at $5/month. You just upload your app., and go.
> >
> > The cheapest way for a company to offer Erlang hosting would be
> > within virtual private servers (VPS), e.g. Linux Virtual Machines
> > running Erlang Nodes. I've googled around and can't find anyone doing
> > that. Of course, you could build the virtual private server yourself,
> > but that seems like significant effort which a hosting company could
> > just 'solve' for people.
> > Is there a pre-built VPS image to take out all of the pain?
> >
> > A nice solution would be to have a "pre-configured, templated image"
> > of an Erlang environment and Erlang/OTP at Amazon Elastic Compute
> > Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html?node=201590011
> > Amazon's computing cloud supports starting up and closing down upnew
> > servers on-demand, you only pay for what you use. This would be very
> > handy if your throughput requirements are very peaky.
> >
> > GB
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/b1528e71/attachment.htm>

From ciprian.craciun@REDACTED  Mon Oct  8 13:23:39 2007
From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun)
Date: Mon, 8 Oct 2007 14:23:39 +0300
Subject: [erlang-questions] Code vs. binary heap storage
In-Reply-To: <o1hcl2notl.fsf@erix.ericsson.se>
References: <B73FAC3A-D103-479B-A430-209ACC1659BE@duomark.com>
	<o1hcl2notl.fsf@erix.ericsson.se>
Message-ID: <8e04b5820710080423r4f04a091w63dac0a970accbf0@mail.gmail.com>

    These questions were very interesting and I have waited with great
interest the answers but...

On 08 Oct 2007 07:43:50 +0200, Bjorn Gustavsson <bjorn@REDACTED> wrote:
> Jay Nelson <jay@REDACTED> writes:
>
> > Given the following code:
> >
> > return_bin(pinocchio) ->  <<"Once upon a time... ">>;
> > return_bin(cinderella) -> <<"In a castle long ago...">>;
> > return_bin(star_wars) -> <<"In a galaxy far, far way...">>.
> >
> > Assume each of the binaries is of some significant size > 32 bytes.
> >
> > If the module containing this function is code-loaded into a node, I
> > want to verify whether the following are true:
> >
> > 1) The binary only exists once regardless of how many processes or
> > modules access the functions.
> > 2) The binary for each branch of the function is stored in the binary
> > heap.
> > 3) The function return is a pointer to the existing binary heap element.
>
> No to all three questions. A new binary is constructed every time the
> code is called, regardless of which process calls it.

    Doesn't this mean that a lot of memory will be allocated for many
calls, even if the values could have been cached and not created at
each call?

> > 4) The memory footprint of a process which can call these functions
> > does not include the size of the binaries, even if the functions are
> > called and a process variable is bound to the value.
>
> Yes, the binary itself will be stored outside of the process (provided
> the size of the binary is > 64).

    Why would you in fact copy an object if the size is smaller than
64 bytes? What is the reason for it?

> > 5) These binaries can never be garbage collected unless the module
> > containing the functions is unloaded (and all other references
> > obtained by calling the functions are released).
>
> No. None of the binaries reference the loaded code.
>
> > 6) Having a process per binary and a process to route requests would
> > not be more or less memory efficient than having a single process
> > with the above function in place of the routing process.
>
> One or or more processes doesn't matter.
>
>
> To share the binaries, you should call return_bin/1 only once for each
> binary and store the result in some sort of dictionary (dict, gb_trees,
> or process dictionary) or ets table, then retrieve the binary from there
> when you need it.

    Wouldn't the lookup be a time consuming operation on the long
term? (I know that the lookup complexity is O(1)... But still there is
a some code involved...)

    Ciprian.


From per@REDACTED  Mon Oct  8 14:42:03 2007
From: per@REDACTED (Per Hedeland)
Date: Mon, 8 Oct 2007 14:42:03 +0200 (CEST)
Subject: [erlang-questions] prim_inet:accept() bug with {fd,
	N} socket option?
In-Reply-To: <470A1DD0.1030703@gmail.com>
Message-ID: <200710081242.l98Cg3HZ011380@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>Per Hedeland wrote:
>
>> I converted the patch to match R11B-5, it was pretty trivial, but let me
>> know if you want it (untested though).
>
>Yes, please. Since I haven't applied it yet, I'd rather start with what 
>you have for R11B-5 to begin with.

Below - it does compile at least (all three times:-).

--Per


--- otp_src_R11B-5/erts/emulator/drivers/common/inet_drv.c.ORIG	2007-10-07 19:13:52.000000000 +0200
+++ otp_src_R11B-5/erts/emulator/drivers/common/inet_drv.c	2007-10-07 19:40:05.000000000 +0200
@@ -4821,9 +4821,17 @@
 }
 #endif
 
+/* Per H @ Tail-f: The original code here had problems that possibly
+   only occur if you abuse it for non-INET sockets, but anyway:
+   a) If the getsockopt for SO_PRIORITY or IP_TOS failed, the actual
+      requested setsockopt was never even attempted.
+   b) If {get,set}sockopt for one of IP_TOS and SO_PRIORITY failed,
+      but ditto for the other worked and that was actually the requested
+      option, failure was still reported to erlang.                  */
+
 #if  defined(IP_TOS) && defined(SOL_IP) && defined(SO_PRIORITY)
 static int setopt_prio_tos_trick
-	   (int fd, int proto, int type, char* arg_ptr, int arg_sz)
+	(int fd, int proto, int type, char* arg_ptr, int arg_sz, int propagate)
 {
     /* The relations between SO_PRIORITY, TOS and other options
        is not what you (or at least I) would expect...:
@@ -4836,6 +4844,8 @@
     int          tmp_ival_prio;
     int          tmp_ival_tos;
     int          res;
+    int          res_prio;
+    int          res_tos;
 #ifdef HAVE_SOCKLEN_T
 	    socklen_t
 #else
@@ -4844,30 +4854,30 @@
 		tmp_arg_sz_prio = sizeof(tmp_ival_prio),
 		tmp_arg_sz_tos  = sizeof(tmp_ival_tos);
 
-    res = sock_getopt(fd, SOL_SOCKET, SO_PRIORITY,
-		      (char *) &tmp_ival_prio, &tmp_arg_sz_prio);
+    res_prio = sock_getopt(fd, SOL_SOCKET, SO_PRIORITY,
+			   (char *) &tmp_ival_prio, &tmp_arg_sz_prio);
+    res_tos = sock_getopt(fd, SOL_IP, IP_TOS, 
+			  (char *) &tmp_ival_tos, &tmp_arg_sz_tos);
+    res = sock_setopt(fd, proto, type, arg_ptr, arg_sz);
     if (res == 0) {
-	res = sock_getopt(fd, SOL_IP, IP_TOS, 
-		      (char *) &tmp_ival_tos, &tmp_arg_sz_tos);
-	if (res == 0) {
-	    res = sock_setopt(fd, proto, type, arg_ptr, arg_sz);
-	    if (res == 0) {
-		if (type != SO_PRIORITY) {
-		    if (type != IP_TOS) {
-			res = sock_setopt(fd, 
-					  SOL_IP, 
-					  IP_TOS,
-					  (char *) &tmp_ival_tos, 
-					  tmp_arg_sz_tos);
-		    }
-		    if (res == 0) {
-			res =  sock_setopt(fd, 
-					   SOL_SOCKET, 
-					   SO_PRIORITY,
-					   (char *) &tmp_ival_prio, 
-					   tmp_arg_sz_prio);
-		    }
-		}
+	if (type != SO_PRIORITY) {
+	    if (type != IP_TOS && res_tos == 0) {
+		res_tos = sock_setopt(fd, 
+				      SOL_IP, 
+				      IP_TOS,
+				      (char *) &tmp_ival_tos, 
+				      tmp_arg_sz_tos);
+		if (propagate)
+		    res = res_tos;
+	    }
+	    if (res == 0 && res_prio == 0) {
+		res_prio = sock_setopt(fd, 
+				       SOL_SOCKET, 
+				       SO_PRIORITY,
+				       (char *) &tmp_ival_prio, 
+				       tmp_arg_sz_prio);
+		if (propagate)
+		    res = res_prio;
 	    }
 	}
     }
@@ -5205,7 +5215,7 @@
 	    return -1;
 	}
 #if  defined(IP_TOS) && defined(SOL_IP) && defined(SO_PRIORITY)
-	res = setopt_prio_tos_trick (desc->s, proto, type, arg_ptr, arg_sz);
+	res = setopt_prio_tos_trick (desc->s, proto, type, arg_ptr, arg_sz, propagate);
 #else
 	res = sock_setopt	    (desc->s, proto, type, arg_ptr, arg_sz);
 #endif
@@ -5718,7 +5728,7 @@
 	    return -1;
 	}
 #if  defined(IP_TOS) && defined(SOL_IP) && defined(SO_PRIORITY)
-	res = setopt_prio_tos_trick (desc->s, proto, type, arg_ptr, arg_sz);
+	res = setopt_prio_tos_trick (desc->s, proto, type, arg_ptr, arg_sz, 1);
 #else
 	res = sock_setopt	    (desc->s, proto, type, arg_ptr, arg_sz);
 #endif


From n.decker@REDACTED  Mon Oct  8 16:11:32 2007
From: n.decker@REDACTED (Decker, Nils)
Date: Mon, 8 Oct 2007 16:11:32 +0200
Subject: [erlang-questions] SNMP Agent Trap receiver
Message-ID: <8981957283CC0F49A8E35A51B03CEECE01ED168F@s-mci-hh-02-nt.extern.studio-hamburg.de>

Hello,

In the excellent snmp module there is a difference between the documentation and the code.

The documentation for send_notification states for the Receiver parameter and the snmp_targets message:
"If there are no targets for which an Inform-Request is sent, Addresses is the empty list []."

In R11B2 (debian stable) the snmp_targets message is not sent if there is no receiver.

The cause seem to be these two clauses in snmpa_trap.erl:
send_v2_trap(_TrapRec, [], _Vbs, _Recv, _NetIf, _SysUpTime) ->
    ok;
    
send_v3_trap(_TrapRec, [], _Vbs, _Recv, _NetIf, _SysUpTime, _ContextName) ->
    ok;

The Address for snmp_targets, got_response and no_response should be a tuple of {IP, UDPPort}.

I get for example this tuple: {[1,3,6,1,6,1,1],[192,168,61,10,19,236]}.

[1,3,6,1,6,1,1] is the Oid of snmpUDPDomain and [192,168,61,10,19,236] is the address in the SnmpUDPAddress textual convention.

Thanks
  Nils Decker



-- 
- MCI alles aus einer Hand -
_____________________________

Nils Decker
Projektierung

Studio Hamburg Media Consult International (MCI) GmbH
Jenfelder Allee 80
22039 Hamburg
phone:  +49 (0)40 66 88 34 37
fax:    +49 (0)40 66 88 52 22
E-mail: n.decker@REDACTED
Web:    www.mci-broadcast.com
Gesch?ftsf?hrung: Ralf Schimmel
Prokuristen: J?rn Denneborg, J?rg Pankow
Amtsgericht Hamburg, HRB 70454


From thomasl_erlang@REDACTED  Mon Oct  8 15:49:22 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 8 Oct 2007 06:49:22 -0700 (PDT)
Subject: [erlang-questions] Code vs. binary heap storage
In-Reply-To: <8e04b5820710080423r4f04a091w63dac0a970accbf0@mail.gmail.com>
Message-ID: <186469.4318.qm@web38813.mail.mud.yahoo.com>


--- Ciprian Dorin Craciun <ciprian.craciun@REDACTED>
wrote:

> > Yes, the binary itself will be stored outside of
> the process (provided
> > the size of the binary is > 64).
> 
>     Why would you in fact copy an object if the size
> is smaller than
> 64 bytes? What is the reason for it?

At a guess, doing so is simply a tradeoff versus
allocating and managing small binaries off the global
binary area. So the motivation would be "we have
observed that system performance usually improves by
doing this".

Best,
Thomas



      ____________________________________________________________________________________
Shape Yahoo! in your own image.  Join our Network Research Panel today!   http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 




From gbulmer@REDACTED  Mon Oct  8 21:15:52 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Mon, 8 Oct 2007 20:15:52 +0100
Subject: [erlang-questions] Amazon Elastic Compute Cloud (was Attracting
	Functional Programmers)
In-Reply-To: <mailman.1.1191837602.26409.erlang-questions@erlang.org>
References: <mailman.1.1191837602.26409.erlang-questions@erlang.org>
Message-ID: <49093FD3-57E3-4ECA-A7A7-EF66E78E4D1E@gmail.com>


> Date: Mon, 8 Oct 2007 10:23:53 +0100 (BST)
> From: "Gordon Guthrie"
> Subject: Re: [erlang-questions] Attracting Functional Programmers
> To: erlang-questions@REDACTED
>
>
>> A nice solution would be to have a "pre-configured, templated image"
>> of an Erlang environment and Erlang/OTP at Amazon Elastic Compute
>> Cloud (Amazon EC2) http://www.amazon.com/gp/browse.html? 
>> node=201590011
>
> Dale and I are working on a single-click install Erlang developers
> platform based on the Ubuntu liveCD that comes preloaded with  
> development
> tools...
>
> This uses the Ubuntu Customization Kit to layer a small slice o'Erlang
> over Dapper Drake.
>
> There are a couple of problems with Erlang/Mnesia under the ECC -  
> the most
> noticable one is how you get your data backed off. We are also  
> working on
> an  Mnesia cluster management console that would allow you to set  
> up 2 or
> three ECC instances and distribute the data - and then do a table  
> dump off
> the Amazon storage cloud on a daily basis...
>
> *Eventually* this stuff will be open sourced...

Kudos to you both, this sounds brilliant!
I realise there are a few things to resolve, but Amazon Electric  
Compute Cloud (EC2) should be a great way to exploit some of Erlang/ 
OTP's unique features.

For those unfamiliar with Amazon EC2, it provides a way to get  
compute capacity on demand, and you only pay for the amount of  
compute capacity that you use (about $0.10/hour, I think). The nice  
thing about EC2 is the granularity of capacity change is down to the  
hour (I know there are lots of other companies, but I never noticed  
anyone with such a simple charging model). So if you have a peak  
during the day, or at a full moon, or whatever, you can get more  
capacity, (they claim up to 1000's of servers), then close some down  
when demand has reduced:
http://www.amazon.com/gp/browse.html?node=201590011
They also have a storage system, S3, which looks like a pretty good  
way to buy storage for your compute cloud. (I keep meaning to use it  
as my system backup).

Sorry to sound like an ad, but EC2 is a lovely idea, and I've never  
felt like burning the energy to make use of it until now.

Erlang/OTP naturally complements Amazon EC2 capacity on demand. The  
cute added value of Erlang/OTP is, of course, you could even choose  
to adjust reliability by starting up on more resources. Amazon should  
love you guys. Ericsson should love you too, as this should be an  
amazing demo for Erlang/OTP, and a useful service for companies  
planning to use Erlang, and who are worried about managing their growth.

Are you planning to offer a hosting service on top of Amazon EC2?

GB


From yinso.chen@REDACTED  Mon Oct  8 21:16:25 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 12:16:25 -0700
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
Message-ID: <779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>

Hi -

Thanks, one more question inline.

On 10/8/07, Chandru <chandrashekhar.mullaparthi@REDACTED> wrote:
>
>
> > gen_fsm can certainly be used to implement the logic behind an
> > e-commerce site where you'd have one instance of the gen_fsm for each
> > customer you are handling. You'd somehow have to map (Cookie to
> > process-id) your HTTP request to a particular instance of the state
> > machine. If you wanted to keep the state machine persistent, you'd
> > have to save it to some database and resurrect it when your next
> > request comes in and your state machine isn't around for whatever
> > reason.


Interesting - are you saying erlang processes can be serialized to say
Mnesia and revive later?  If that's doable it's certainly a very intriguing
technique.  Can you point me to some examples/guides on how this is done?

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/1a40f217/attachment.htm>

From chandrashekhar.mullaparthi@REDACTED  Mon Oct  8 22:12:02 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Mon, 8 Oct 2007 21:12:02 +0100
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
	<779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
Message-ID: <b447a30d0710081312o242649a2l6f78d892cbb8a1b6@mail.gmail.com>

On 08/10/2007, YC <yinso.chen@REDACTED> wrote:
> Hi -
>
> Thanks, one more question inline.
>
> On 10/8/07, Chandru <chandrashekhar.mullaparthi@REDACTED >
> wrote:
> >
> > > gen_fsm can certainly be used to implement the logic behind an
> > > e-commerce site where you'd have one instance of the gen_fsm for each
> > > customer you are handling. You'd somehow have to map (Cookie to
> > > process-id) your HTTP request to a particular instance of the state
> > > machine. If you wanted to keep the state machine persistent, you'd
> > > have to save it to some database and resurrect it when your next
> > > request comes in and your state machine isn't around for whatever
> > > reason.
>
> Interesting - are you saying erlang processes can be serialized to say
> Mnesia and revive later?  If that's doable it's certainly a very intriguing
> technique.  Can you point me to some examples/guides on how this is done?
>

It isn't quite what  you are imagining it to be :-)

Assuming you identify a user's HTTP session by means of a cookie and
you start a state machine to guide the customer through a payment
process. Each time your state machine's State gets updated, you could
save it mnesia with the Cookie as the key. The next time another
request comes along, you can spawn a new state machine process which
initialises itself with the state in mnesia and then handles the
request.

There is a way to put a process to sleep so that it doesn't consume
too much resources  - see erlang:hibernate/3. But this method won't
survive a node restart.

cheers
Chandru


From ulf@REDACTED  Mon Oct  8 22:16:46 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Mon, 8 Oct 2007 22:16:46 +0200
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
	<779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
Message-ID: <8209f740710081316l3240b83akdd8cfb5488600706@mail.gmail.com>

2007/10/8, YC <yinso.chen@REDACTED>:
>
> Interesting - are you saying erlang processes can be serialized to say
> Mnesia and revive later?  If that's doable it's certainly a very intriguing
> technique.  Can you point me to some examples/guides on how this is done?

You can't save the actual process, but you can pretty easily save the
state data of the process. The thing to keep in mind is that PIDs will
change, since it's actually a brand new process after revival.

If your process is a gen_server, the thing that needs saving could
be the whole state variable; for a gen_fsm, it might be the {State, Data}
tuple. You may want to write pack()/unpack() functions that compress
the state somewhat, e.g. removing things that aren't needed for
restoring the process later.

BR,
Ulf W


From yinso.chen@REDACTED  Mon Oct  8 22:37:23 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 13:37:23 -0700
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <b447a30d0710081312o242649a2l6f78d892cbb8a1b6@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
	<779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
	<b447a30d0710081312o242649a2l6f78d892cbb8a1b6@mail.gmail.com>
Message-ID: <779bf2730710081337rc088418h27efe6ecf0ce0fcf@mail.gmail.com>

On 10/8/07, Chandru <chandrashekhar.mullaparthi@REDACTED> wrote:
>
> > It isn't quite what  you are imagining it to be :-)
>
> > Assuming you identify a user's HTTP session by means of a cookie and
> > you start a state machine to guide the customer through a payment
> > process. Each time your state machine's State gets updated, you could
> > save it mnesia with the Cookie as the key. The next time another
> > request comes along, you can spawn a new state machine process which
> > initialises itself with the state in mnesia and then handles the
> > request.


Haha - I see.  That's a much more familiar process for me, although if
process can be serialized that would be quite interesting indeed.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/3e2263a1/attachment.htm>

From yinso.chen@REDACTED  Mon Oct  8 22:39:06 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 8 Oct 2007 13:39:06 -0700
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <8209f740710081316l3240b83akdd8cfb5488600706@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
	<779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
	<8209f740710081316l3240b83akdd8cfb5488600706@mail.gmail.com>
Message-ID: <779bf2730710081339o441847b8pdf7a91b3a1901a62@mail.gmail.com>

On 10/8/07, Ulf Wiger <ulf@REDACTED> wrote:
>
> > You can't save the actual process, but you can pretty easily save the
> > state data of the process. The thing to keep in mind is that PIDs will
> > change, since it's actually a brand new process after revival.
>
> > If your process is a gen_server, the thing that needs saving could
> > be the whole state variable; for a gen_fsm, it might be the {State,
> Data}
> > tuple. You may want to write pack()/unpack() functions that compress
> > the state somewhat, e.g. removing things that aren't needed for
> > restoring the process later.


Thanks.

Where can I find the definition of the State variable to figure out what's
needed and not needed?

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071008/02223547/attachment.htm>

From alexander.lamb@REDACTED  Tue Oct  9 07:50:41 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Tue, 9 Oct 2007 07:50:41 +0200
Subject: [erlang-questions] Mnesia table events?
Message-ID: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>

Hello List,

Ok, I'll go for the short question:-)

Is it possible to use Mnesia table events to get notified of all data  
changes in a production system? (or are table events only used for  
debugging or other testing features).
Is there a performance risk associated with having one or several  
listeners?

Thanks,

Alex
--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071009/d59a55b8/attachment.htm>

From chandrashekhar.mullaparthi@REDACTED  Tue Oct  9 08:24:17 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Tue, 9 Oct 2007 07:24:17 +0100
Subject: [erlang-questions] Mnesia table events?
In-Reply-To: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>
References: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>
Message-ID: <b447a30d0710082324n640b9103u965cf6650eb2d3cb@mail.gmail.com>

On 09/10/2007, Alexander Lamb <alexander.lamb@REDACTED> wrote:
> Hello List,
>
> Ok, I'll go for the short question:-)
>
> Is it possible to use Mnesia table events to get notified of all data
> changes in a production system? (or are table events only used for debugging
> or other testing features).
> Is there a performance risk associated with having one or several listeners?

I've used mnesia table events in a production system. I have two
"listeners" and the rate of updates to the database is probably about
10/sec max. I haven't tested it under heavy loads to see how it would
scale. I guess the only way for you to find out is to prototype it.

cheers
Chandru


From bengt.kleberg@REDACTED  Tue Oct  9 09:23:11 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Tue, 09 Oct 2007 09:23:11 +0200
Subject: [erlang-questions] OT: Re: Surprising conclusion in .
 Parallel	programming environments: less is more
In-Reply-To: <4706E615.1000504@ghostgun.com>
References: <470349D7.2050108@ericsson.com>	<40F0D9E5-73AF-45F6-BBEB-44AE11096ED0@cs.otago.ac.nz>	<015e01c806ad$bff25de0$891ea8c0@SSI.CORP>	<4705DE91.2080004@ericsson.com>
	<4706E615.1000504@ghostgun.com>
Message-ID: <470B2C5F.8060500@ericsson.com>

actually, the truth is that sweden is such an utopia that we do not want 
other people to come here and then refuse to leave. so we have 
deliberately made it as hard as possible to learn swedish.


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-06 03:34, jm wrote:
> Heading off topic here....

...deleted
> So native Swedish speakers here's you chance to promote the learning of 
> Swedish. Speak up and recommend good free Swedish learn material 
> available online.



From erlang@REDACTED  Tue Oct  9 09:57:53 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Tue, 9 Oct 2007 09:57:53 +0200
Subject: [erlang-questions] John Hughes at IFCP - quickcheck + erlang video
Message-ID: <9b08084c0710090057q2d041037o6c457c7a43705346@mail.gmail.com>

John Hughes talked about quickchecking  Erlang (and other languages) at ICFP.

I saw John live at ICFP - it was a great talk and is now on google videos

http://video.google.com/videoplay?docid=4655369445141008672

/Joe Armstrong


From erlang@REDACTED  Tue Oct  9 10:20:58 2007
From: erlang@REDACTED (Peter Lund)
Date: Tue, 09 Oct 2007 10:20:58 +0200
Subject: [erlang-questions] gen_tcp custom framing
In-Reply-To: <9e009ad0710080406k5e0ba800q8e9721dc1f68896a@mail.gmail.com>
References: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>	
	<47064E36.708@lundata.se>
	<9e009ad0710080406k5e0ba800q8e9721dc1f68896a@mail.gmail.com>
Message-ID: <470B39EA.60408@lundata.se>

Joern skrev:
>> In R9, I have also forced to match each packet more than one time since
>> an initial length value read out first, cannot be used later in the same
>> binary match statement, but I am guessing that this problem now is history
>> when using R11.
>>     
>
> Hmm - I'm not sure I follow you on this one. Can you elaborate?
>   

I think I was overly hopeful on what R11 can do (working daily with R9C).

1> f(), <<Z,D:Z/binary,R/binary>> = <<2,3,4,5,6,7,8,9,0 >>, {Z,D,R}.
{2,<<3,4>>,<<5,6,7,8,9,0>>}

2> f(), <<Z,D:Z+1/binary,R/binary>> = <<2,3,4,5,6,7,8,9,0 >>, {Z,D,R}.
** 1: syntax error before: '+' **

I thought/hoped that 2 above would work, but when testing it, it did not.

The size field read into Z from the first octet could be used as the 
size of next match in the binary. (1)

But the size could NOT be an expression, in this case involving Z (2).

This forces you to do sequencial binary matching:
 - read size Z,
 - assign Z2 = expression(Z)
 - read rest of the binary using Z2

/Peter




From hakan@REDACTED  Tue Oct  9 10:36:51 2007
From: hakan@REDACTED (Hakan Mattsson)
Date: Tue, 9 Oct 2007 10:36:51 +0200 (CEST)
Subject: [erlang-questions] Mnesia system events and table events
In-Reply-To: <FEAC09FC-8AE9-4427-A6B0-400BC07C3C0B@rodanotech.ch>
References: <FEAC09FC-8AE9-4427-A6B0-400BC07C3C0B@rodanotech.ch>
Message-ID: <Pine.LNX.4.64.0710090959130.25159@amrod.du.uab.ericsson.se>

On Sun, 7 Oct 2007, Alexander Lamb wrote:

AL> Is this the way to go? In details:
AL> 
AL> 1) is using table events in Mnesia a usual way to notify other processes of
AL> changes?

Yes.

AL> 2) I suppose the events are sent aynchronously... what about performance
AL> issues?

Yes, events are sent asynchronously. They are just
plain Erlang messages. In order to ensure that the
performance is good enough for your configuration and
access patterns you need to perform some measurements
yourself.

AL> 3) If I have a multi-node Mnesia database, I suppose I will only receive one
AL> event per event (not one per node)?

No. The subscription mechanism in Mnesia is a local feature.
When a table is updated, Mnesia will update all table
replicas and for each replica a local message will be
sent to the local subscribers.

AL> 4) If this is the way to go, would you send on the bus:
AL> 	a) the Name and Key of the record(s) which changed (leaving the
AL> listening, non Erlang process to refetch immediately after what it needs)
AL> 	b) the complete record(s), event the attributes which didn't change
AL> 	c) only the changed attributes

B. See the documentation:

  http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html#5.7

AL> I was thinking of converting the events in JSON messages which could be
AL> listened to from any non-Erlang process.

How do you to take care of updates that don't make it
all the way to your external non Erlang process?

For example, events may be lost if the Erlang node
crashes after Mnesia has committed the update to the
database but before your external non-Erlang process
has handled it.

/H?kan

From tobbe@REDACTED  Tue Oct  9 15:58:01 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Tue, 09 Oct 2007 15:58:01 +0200
Subject: [erlang-questions] Distrobuted/Fragmented Mnesia node crash
In-Reply-To: <009401c804f1$d59b66e0$80d234a0$@com>
References: <fdtc5e$ec0$1@sea.gmane.org> <009401c804f1$d59b66e0$80d234a0$@com>
Message-ID: <feg1d9$g7j$1@sea.gmane.org>

What errors do you get?

We know there is a lurking bug somewhere in the dets code.
We have got 'bad object' and 'premature eof' every other
month the last year. We have not been able to track the
bug down since the dets files is repaired automatically
next time it is opened.

As a temporary solution we restart mnesia whenever this error occurs...

Btw: it would be very nice if you had the time to write a little
trapexit-tutorial describing your fragmented setup.

Cheers, Tobbe

Eranga Udesh wrote:
> Hi,
> 
> I run 3 distributed Erlang nodes, dedicated to run Mnesia database server.
> Another Erlang node runs the application which read/write to those tables.
> Schema is created as disk_copies in all 4 nodes and tables are created as
> fragmented and distributed tables between 3 dedicated Mnesia nodes. I.e. 99
> fragments of a table distributed in 3 nodes resulting 33 fragments in each.
> There're about 250-350 Db write/s and 500-800 DB read/s
> 
> Performance is quite good. I don't see "Mnesia overloaded" warnings. However
> occasionally one of the Mnesia node crashes. Sometimes the time between 2
> crashes of the same Mnesia node may be about 1-2 days, while the next crash
> takes about 10-20 days to happen. It doesn't even generate the crash_dump
> file or even if its generated, the size is 0 bytes.
> 
> Any idea what could be the cause? Are others also experience the same? I
> don't see any abnormal Load, Processor or Memory usage when these crashes
> occurs.
> 
> BRgds,
> - Eranga
> 
> 



From bbrown@REDACTED  Tue Oct  9 16:43:14 2007
From: bbrown@REDACTED (bbrown)
Date: Tue, 9 Oct 2007 10:43:14 -0400
Subject: [erlang-questions] My hello world erlang app crashed
Message-ID: <20071009143913.M62298@www.botspiritcompany.com>

I feel silly for asking.  Normally I have been compiling code from the shell 
and decided to launch a script as a standalone.  I took this code from the 
programming erlang book and got the following error (on Win32, erlang v5.5.5)

erl -noshell -s hello start -s init stop
(only a hello.erl file exists)

Hmm, Actually, I compiled it first and it worked.

 erlc hello.erl
 erl -noshell -s hello start -s init stop

--
Berlin Brown
[berlin dot brown at gmail dot com]
http://botspiritcompany.com/botlist/?



From joelr1@REDACTED  Tue Oct  9 17:40:49 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 9 Oct 2007 16:40:49 +0100
Subject: [erlang-questions] erl_errno on the Mac
Message-ID: <182DDFB1-1C56-4AB1-8E5A-F6309B2AB82F@gmail.com>

Does anyone else have problems with erl_errno on the Mac?

It's not expanded unless _REENTRANT is set and it does not appear to  
be set by default.

I defined _REENTRANT manually but I wonder if it's gonna come back to  
bite me later.

	Thanks, Joel

--
http://wagerlabs.com







From joelr1@REDACTED  Tue Oct  9 18:15:11 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 9 Oct 2007 17:15:11 +0100
Subject: [erlang-questions] Troubleshooting ei_xreceive_msg
Message-ID: <4B6479B9-0C8A-47F0-B9E2-397D45E4EF49@gmail.com>

I can connect to my Erlang node like this:

	ei_x_new_with_version(&x_in);
	ei_x_new_with_version(&x_out);
	
	if (ei_connect_init(&ec, "foo", "secretcookie", n++) < 0)
	{
     NSLog(@"ERROR when initializing: %d", erl_errno);
     exit(-1);
	}

but then each call to ei_xreceive_msg results in ERL_ERROR and  
erl_errno is set to 5.

I think that's EIO but I can't figure out the meaning of the error in  
the context of receiving the message.

Any suggestions?

	Thanks, Joel

--
http://wagerlabs.com







From garry@REDACTED  Tue Oct  9 18:24:36 2007
From: garry@REDACTED (Garry Hodgson)
Date: Tue, 9 Oct 2007 12:24:36 -0400 (EDT)
Subject: [erlang-questions] network outage recovery, again
Message-ID: <2007100912241191947076@k2.sage.att.com>

(sorry for the length of this.  just trying to provide enough context)

i've got an app that i'm trying to make more robust in the face of 
some recent network outages, as discussed previously:
http://www.erlang.org/pipermail/erlang-questions/2007-June/027535.html

a proposed solution was to have the compute nodes check in periodically.
so i run the nodes something like this:

   erl -sname node -run node oam_start node@REDACTED -detached

where node:oam_start is:

    oam_start( _ ) ->
        checkin(),
        timer:apply_interval( ?HeartbeatInterval, node, checkin, [] ).

    checkin() -> { master, masternode() } ! { checkin, node() }

to test this, i pull the plug on the node, wait about a minute, and
then the master gets a nodedown message.  plug it back in and wait...
nothing.  check erlang on the node, and it's not running.

if i run it on the node without the -detached flag, and start by hand,
it works ok:

    Eshell V5.3.6.3  (abort with ^G)
    (node@REDACTED)1> node:oamStart( [] ).
    Node node@REDACTED is checking in to fathom@REDACTED
    {ok,{interval,#Ref<0.0.0.49>}}
    (node@REDACTED)2> Node node@REDACTED is checking in to fathom@REDACTED
    Node node@REDACTED is checking in to fathom@REDACTED

    =ERROR REPORT==== 3-Oct-2007::15:11:44 ===
    ** Node fathom@REDACTED not responding **
    ** Removing (timedout) connection **

    Node node@REDACTED is checking in to fathom@REDACTED
    Node node@REDACTED is checking in to fathom@REDACTED
    Node node@REDACTED is checking in to fathom@REDACTED

so it works within the erlang shell, but not detached.  i assumed
this meant that there was some exception that the shell was catching,
but that, uncaught, caused erlang to go away when run with -detached.
so i wrapped the message send in a catch().  no dice.  same problem.

so, question 1:
what is happening in the above to cause the node to die?

what can i do to ensure that the node doesn't die, but keeps trying
until the network heals itself (typically a few minutes)?

question 2:
in trying to figure out this problem, i noticed that i never got
a nodeup message when the node starts.  so i added a call to
net_kernel:monitor_nodes(true), and now i do (i had previously
had the master ping each of its nodes, and call monitor_node()
on all that respond.  this caused me to get the nodedown messages
that master used to monitor its resources, but no nodeup messages).  

this behavior makes me wonder if there's any reason for the above
mechanism at all.  that is, can i just have master call monitor_nodes(true),
and then rely on nodeup messages to tell me when a node is back?  will
i get one of these if the network heals?  or just when a node actually
starts?

thanks for any insight you can provide.


----
Garry Hodgson, Senior Software Geek, AT&T CSO

nobody can do everything, but everybody can do something.
do something.



From peppe@REDACTED  Tue Oct  9 18:40:56 2007
From: peppe@REDACTED (Peter Andersson)
Date: Tue, 09 Oct 2007 18:40:56 +0200
Subject: [erlang-questions] erl_errno on the Mac
In-Reply-To: <182DDFB1-1C56-4AB1-8E5A-F6309B2AB82F@gmail.com>
References: <182DDFB1-1C56-4AB1-8E5A-F6309B2AB82F@gmail.com>
Message-ID: <470BAF18.7030701@erix.ericsson.se>

Hi Joel,

We have very recently corrected a bug where the __erl_errno symbol was
undefined in the single thread version of the ei library. It sounds like
this is the problem you are having. Since we are not making any more R11
releases now, the fix will not be available until after the R12 release.
However, if you add the following to the very end of
lib/erl_interface/src/misc/ei_pthreads.c and rebuild libei_st.a, you
should be able to link to the lib without problems.

#if !defined(_REENTRANT) && !defined(VXWORKS) && !defined(__WIN32__)
volatile int __erl_errno;
#endif

Best regards,
Peter

Erlang/OTP, Ericsson


Joel Reymont wrote:
> Does anyone else have problems with erl_errno on the Mac?
>
> It's not expanded unless _REENTRANT is set and it does not appear to  
> be set by default.
>
> I defined _REENTRANT manually but I wonder if it's gonna come back to  
> bite me later.
>
> 	Thanks, Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>   



From casper2000a@REDACTED  Tue Oct  9 19:01:27 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Tue, 9 Oct 2007 22:31:27 +0530
Subject: [erlang-questions] Distrobuted/Fragmented Mnesia node crash
In-Reply-To: <feg1d9$g7j$1@sea.gmane.org>
References: <fdtc5e$ec0$1@sea.gmane.org> <009401c804f1$d59b66e0$80d234a0$@com>
	<feg1d9$g7j$1@sea.gmane.org>
Message-ID: <001801c80a96$0a10bb60$1e323220$@com>

Well, the node suddenly terminates. I think getting Segmentation Fault. Node
is running as a Release and I used both ERL_CRASH_DUMP and
ERL_CRASH_DUMP_SECONDS env variables, but unfortunately the crash even
doesn't create the Crash Dump file. I will try to debug this further and
publish any findings I get. Meanwhile anybody else getting a similar issues,
pls share.

Yes, I will do a write up at trapexit about fragmented mnesia.

Brgds,
- Eranga




-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torbjorn Tornkvist
Sent: Tuesday, October 09, 2007 7:28 PM
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Distrobuted/Fragmented Mnesia node crash

What errors do you get?

We know there is a lurking bug somewhere in the dets code.
We have got 'bad object' and 'premature eof' every other
month the last year. We have not been able to track the
bug down since the dets files is repaired automatically
next time it is opened.

As a temporary solution we restart mnesia whenever this error occurs...

Btw: it would be very nice if you had the time to write a little
trapexit-tutorial describing your fragmented setup.

Cheers, Tobbe

Eranga Udesh wrote:
> Hi,
> 
> I run 3 distributed Erlang nodes, dedicated to run Mnesia database server.
> Another Erlang node runs the application which read/write to those tables.
> Schema is created as disk_copies in all 4 nodes and tables are created as
> fragmented and distributed tables between 3 dedicated Mnesia nodes. I.e.
99
> fragments of a table distributed in 3 nodes resulting 33 fragments in
each.
> There're about 250-350 Db write/s and 500-800 DB read/s
> 
> Performance is quite good. I don't see "Mnesia overloaded" warnings.
However
> occasionally one of the Mnesia node crashes. Sometimes the time between 2
> crashes of the same Mnesia node may be about 1-2 days, while the next
crash
> takes about 10-20 days to happen. It doesn't even generate the crash_dump
> file or even if its generated, the size is 0 bytes.
> 
> Any idea what could be the cause? Are others also experience the same? I
> don't see any abnormal Load, Processor or Memory usage when these crashes
> occurs.
> 
> BRgds,
> - Eranga
> 
> 

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions




From joelr1@REDACTED  Tue Oct  9 19:04:12 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 9 Oct 2007 18:04:12 +0100
Subject: [erlang-questions] erl_errno on the Mac
In-Reply-To: <470BAF18.7030701@erix.ericsson.se>
References: <182DDFB1-1C56-4AB1-8E5A-F6309B2AB82F@gmail.com>
	<470BAF18.7030701@erix.ericsson.se>
Message-ID: <008107DA-E6D4-4E16-9A6B-F8A403886CF9@gmail.com>

Peter,

On Oct 9, 2007, at 5:40 PM, Peter Andersson wrote:

> #if !defined(_REENTRANT) && !defined(VXWORKS) && !defined(__WIN32__)
> volatile int __erl_errno;
> #endif

I'm linking against libei.a, should I be using libei_st.a instead?

	Thanks, Joel

--
http://wagerlabs.com







From tty.erlang@REDACTED  Wed Oct 10 00:34:07 2007
From: tty.erlang@REDACTED (t ty)
Date: Tue, 9 Oct 2007 18:34:07 -0400
Subject: [erlang-questions] Erlang Workshop 2007
Message-ID: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>

It has been 4 days and Google still hasn't found any blog posts
regarding the workshop. You guys must be busy working :)

Here is my quick write up, hopefully enough to interest folks to read
the papers and in attending EUC this year and the 2008 Erlang
Workshop. Unfortunately I could not stay for the discussion, perhaps
someone else could fill in the blanks.

Disclaimer: omission and mistakes are my own.

Regards

t
-------------- next part --------------
A non-text attachment was scrubbed...
Name: erlangworkshop2007
Type: application/octet-stream
Size: 3846 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071009/33ba93d8/attachment.obj>

From dbudworth@REDACTED  Wed Oct 10 03:27:59 2007
From: dbudworth@REDACTED (David Budworth)
Date: Tue, 9 Oct 2007 20:27:59 -0500
Subject: [erlang-questions] Additional question regarding: gen_server vs
	gen_fsm
Message-ID: <2e23d1d20710091827n16f585f5t95985e1824205116@mail.gmail.com>

Hi folks,

YC's topic titled gen_server vs gen_fsm got me wondering

Note for the reader: Sorry, just realized this is really long, but I'm
really curious as to how/if this can be solved.  The last 3/4 of this is
more bouncing ideas off people.

I'm investigating writing an order routing system using erlang and can
imagine a few different ways to deal with it (also an erlang newbie)

in our case, the duration of a transaction can be months.
So I'm assuming we really do need to persist state via mnesia (which I'll
bug people about in upcoming messages, I'm sure).

aside from that, there's a natural state progression for an order on an
exchange, they tend to have states that go from:
new -> confirmed -> partial_fill -> filled
(and canceled, expired, etc depending on what really happens).

At first glance of gen_fsm vs gen_server, it seems like gen_fsm is really a
fancyish tracking mechanism built on top of (or really similar to)
gen_server to keep state transition plumbing out of the 'business logic'
area.


so basically, the idea is (just looking for confirmation):
Current State: confirmed

confirmed(FillEvent,StateData) -> signal_done, {done,DoneStateData};
confirmed(PartialFillEvent,StateData) -> {partial_fill,UpdatedStateData}.

partial_fill(FillEvent,StateData) -> signal_done(), {done,DoneStateData}.

something to that effect.

so the question is, it seems like I'd have to manually persist "StateData"
on each event handled to survive process/host failures and basically when
detecting a process failure I'd have to spawn a new gen_fsm and register it
(say with order id or something) passing in the most recently persisted
state.

Am I on the right track?

I guess the only thing odd here, or rather, what I'm not entirely sure how
it's handled is that with everything happening async in general, how does
one guarantee that a message isn't lost while in transit and a host fails?

coming from the JMS world, you receive a message in a TX, then do your work,
publish a new message in that TX then commit.  If you die in the middle, the
message is re-delivered to another listener sitting there.

in the OTP world, what is the equivalent to this?
meaning:
SomeProcessThatSendsAnEvent:

OrderHandlerPID ! MessageToProcess.

OrderHandler
   receive
     Msg ->  start_doing_stuff(), crash().

it seems like the only way to guarantee delivery is to make everything
Request/Response synchronized.
so you send a message then wait for an ack.  And if it doesn't come, you
lookup/respawn/whatever a pid to re-send to until it succeeds.

So if a process takes a long time, I'm blocking the sender.   Whereas in JMS
(or JavaSpaces really) I know that if I send a message, that the message
broker will Never(tm) lose the message and it will eventually be handled.

Only way I see to make fast senders and reliable delivery would be to send a
message to an intermediate process that persists/replicates to a buddy, and
have the consumers send "Message Consume Requests" to that same intermediate
process which will deliver the message and mark/delete it (still with a race
condition since I may have deleted before sending or sent before deleting)
giving me dups.

it's almost as though what I really need is to use mnesia as my message bus
so the "consumer" will do a select, process and submit an update.  This only
works if mnesia supports operations similar to:
begin tx
insert (response values) in to msg table
update (request row with DONE status) table where (request row) exists %% or
delete the row, either way
commit

so I can put the response back if and only if I'm the first to mark it
done.  This is in the event that 2 consumers attempt to process the same
message.

I'm basically trying to figure out if there's something similar to the
Once And Only Once delivery semantic you get with JMS, or if you have to be
willing to take a message loss.

now, our current stuff is all java based, and I'm expecting to use (haven't
tried yet) JInterface to bridge the gap between our java stuff and new
erlang stuff.

Is this type of thing just not what erlang is meant for?  Or am I just
missing the super_duper_jms_like module?


-David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071009/70781911/attachment.htm>

From yuxh312@REDACTED  Wed Oct 10 03:42:31 2007
From: yuxh312@REDACTED (krzycube)
Date: Wed, 10 Oct 2007 09:42:31 +0800
Subject: [erlang-questions] Erlang Workshop 2007
In-Reply-To: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>
Message-ID: <e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>

thx , tty
and , where to download these papers?

---krzycube

On 10/10/07, t ty <tty.erlang@REDACTED> wrote:
>
> It has been 4 days and Google still hasn't found any blog posts
> regarding the workshop. You guys must be busy working :)
>
> Here is my quick write up, hopefully enough to interest folks to read
> the papers and in attending EUC this year and the 2008 Erlang
> Workshop. Unfortunately I could not stay for the discussion, perhaps
> someone else could fill in the blanks.
>
> Disclaimer: omission and mistakes are my own.
>
> Regards
>
> t
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>


-- 
"Yu Xihe" as Chinese Name
"Martin" as English Name
"KrzyCube" as Nickname
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/6cde8654/attachment.htm>

From tty.erlang@REDACTED  Wed Oct 10 03:52:11 2007
From: tty.erlang@REDACTED (t ty)
Date: Tue, 9 Oct 2007 21:52:11 -0400
Subject: [erlang-questions] Erlang Workshop 2007
In-Reply-To: <e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>
	<e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>
Message-ID: <290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>

The papers should be part of the ACM SIGPLAN library. Guess you have
to go thru them. Local university library is another good choice :)

Perhaps the individual authors can make their papers available. I'm
not sure what the agreement with ACM is like.

t

On 10/9/07, krzycube <yuxh312@REDACTED> wrote:
> thx , tty
> and , where to download these papers?
>
> ---krzycube
>
>
> On 10/10/07, t ty <tty.erlang@REDACTED > wrote:
> >
> > It has been 4 days and Google still hasn't found any blog posts
> > regarding the workshop. You guys must be busy working :)
> >
> > Here is my quick write up, hopefully enough to interest folks to read
> > the papers and in attending EUC this year and the 2008 Erlang
> > Workshop. Unfortunately I could not stay for the discussion, perhaps
> > someone else could fill in the blanks.
> >
> > Disclaimer: omission and mistakes are my own.
> >
> > Regards
> >
> > t
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> >
>
>
>
> --
> "Yu Xihe" as Chinese Name
> "Martin" as English Name
> "KrzyCube" as Nickname


From ulf@REDACTED  Wed Oct 10 05:11:07 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Wed, 10 Oct 2007 05:11:07 +0200
Subject: [erlang-questions] Erlang Workshop 2007
In-Reply-To: <290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>
	<e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>
	<290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
Message-ID: <8209f740710092011w1d6195d9i2e3b0d856e7dcda2@mail.gmail.com>

The gproc paper and code can be found at

svn co http://svn.ulf.wiger.net/gproc

BR,
Ulf W

2007/10/10, t ty <tty.erlang@REDACTED>:
> The papers should be part of the ACM SIGPLAN library. Guess you have
> to go thru them. Local university library is another good choice :)
>
> Perhaps the individual authors can make their papers available. I'm
> not sure what the agreement with ACM is like.
>
> t
>
> On 10/9/07, krzycube <yuxh312@REDACTED> wrote:
> > thx , tty
> > and , where to download these papers?
> >
> > ---krzycube
> >
> >
> > On 10/10/07, t ty <tty.erlang@REDACTED > wrote:
> > >
> > > It has been 4 days and Google still hasn't found any blog posts
> > > regarding the workshop. You guys must be busy working :)
> > >
> > > Here is my quick write up, hopefully enough to interest folks to read
> > > the papers and in attending EUC this year and the 2008 Erlang
> > > Workshop. Unfortunately I could not stay for the discussion, perhaps
> > > someone else could fill in the blanks.
> > >
> > > Disclaimer: omission and mistakes are my own.
> > >
> > > Regards
> > >
> > > t
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> > >
> >
> >
> >
> > --
> > "Yu Xihe" as Chinese Name
> > "Martin" as English Name
> > "KrzyCube" as Nickname
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From berlin.brown@REDACTED  Wed Oct 10 05:19:55 2007
From: berlin.brown@REDACTED (Berlin Brown)
Date: Tue, 9 Oct 2007 23:19:55 -0400
Subject: [erlang-questions] Can't start yaws with startup provided startup
	script
Message-ID: <df27a55b0710092019h111c1dbcjf4225929b675645f@mail.gmail.com>

I am trying to start yaws with the init.d script provided and I get
the following error on startup.  But when I launch from the
command-line, it works fine.  I basically used everything out of the
box and updated the RC scripts to include the yaws daemon.

I can start it manually, I guess for now.

boot.log

Oct  9 23:04:27  yaws: Eshell V5.5.5  (abort with ^G)
Oct  9 23:04:27 yaws: 1>
Oct  9 23:04:27 yaws: ** Terminating erlang (nonode@REDACTED) **
Oct  9 23:04:27 yawsd: yaws startup succeeded

report.log

=ERROR REPORT==== 9-Oct-2007::22:56:12 ===
** Generic server <0.20.0> terminating
** Last message in was {'EXIT',<0.22.0>,eio}
** When Server state == {state,user_sup,
                               undefined,
                               <0.22.0>,
                               {<0.20.0>,user_sup}}
** Reason for termination ==
** eio

Config
<server servername.com>
        port = 2401
        listen = 0.0.0.0
        docroot = /usr/local/var/yaws/www
</server>


-- 
Berlin Brown
http://botspiritcompany.com/botlist/spring/help/about.html
newspirit technologies


From findraju@REDACTED  Wed Oct 10 07:09:14 2007
From: findraju@REDACTED (Raj)
Date: Wed, 10 Oct 2007 10:39:14 +0530
Subject: [erlang-questions] inet_dns:dns_exp, crash in erlang
Message-ID: <57e317240710092209g1842b2fep802b7a207eaca918@mail.gmail.com>

Hi all,

I have got crash in erlang when I continuously send dns_queries to
the dns_server. The crash I found is in inet_dns:dn_exp in
otp/lob/kernel/src/inet_dns.erl
The log is pasted below,

== 2-Mar-2007::15:55:42 == INFO REPORT - info <0.24575.0>

{dns_query,145,

false,

"Invalid query from {{10,56,210,2},53,udp} - {error,fmt}"}

== 2-Mar-2007::15:55:42 == INFO REPORT - info <0.24578.0>

{dns_query,145,

false,

"Invalid query from {{10,56,210,2},53,udp} - {error,fmt}"}

== 2-Mar-2007::15:55:43 == INFO REPORT - info <0.24580.0>

{dns_query,145,

false,

"Invalid query from {{10,56,210,2},53,udp} - {error,fmt}"}

== 2-Mar-2007::15:55:45 == ERROR - <0.24627.0>

** State machine <0.24627.0> terminating

** Last event in was timeout

** When State == decode

** Data == {state,[254,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

{{10,56,210,2},53,udp},

undefined,

undefined,

undefined,

undefined,

undefined,

0,

undefined,

undefined,

{options,false,false,false,false,false},

{10,56,40,100},

<0.481.0>}

** Reason for termination =

** {function_clause,[{inet_dns,dn_exp,

[[100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[254,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[]]},

{inet_dns,decode_query_section,4},

{inet_dns,decode_sections,6},

{inet_dns,decode,1},

{dns_query,decode,2},

{gen_fsm,handle_msg,7},

{proc_lib,init_p,5}]}

== 2-Mar-2007::15:55:45 == ERROR REPORT - crash_report <0.24627.0>

[[{pid,<0.24627.0>},

{registered_name,[]},

{error_info,{function_clause,[{inet_dns,

dn_exp,

[[100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[254,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[]]},

{inet_dns,decode_query_section,4},

{inet_dns,decode_sections,6},

{inet_dns,decode,1},

{dns_query,decode,2},

{gen_fsm,handle_msg,7},

{proc_lib,init_p,5}]}},

{initial_call,{gen,init_it,

[gen_fsm,

<0.481.0>,

<0.481.0>,

dns_query,

[[254,

0,

0,

0|

<<1,0,0,0,0,0,0,7,100,110,115,116,111,111,108,11,99,111,100,101,110,111,109,105,99,111,110,5,108,111,99,97,108,0,0,1,0,1>>],

{10,56,210,2},

53,

udp,

{options,false,false,false,false,false},

{10,56,40,100},

<0.481.0>],

[]]}},

{ancestors,[<0.481.0>,dns_server,dns_sup,<0.469.0>]},

{messages,[]},

{links,[<0.481.0>]},

{dictionary,[]},

{trap_exit,true},

{status,running},

{heap_size,987},

{stack_size,21},

{reductions,214}],

[]]



Can anybody help me  in sorting out this issue.

Thanks,

Rajukrishnan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/7f3e0940/attachment.htm>

From yuxh312@REDACTED  Wed Oct 10 06:46:11 2007
From: yuxh312@REDACTED (krzycube)
Date: Wed, 10 Oct 2007 12:46:11 +0800
Subject: [erlang-questions] Erlang Workshop 2007
In-Reply-To: <8209f740710092011w1d6195d9i2e3b0d856e7dcda2@mail.gmail.com>
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>
	<e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>
	<290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
	<8209f740710092011w1d6195d9i2e3b0d856e7dcda2@mail.gmail.com>
Message-ID: <e3e496f90710092146n4469fddciab24f1f33f2559e0@mail.gmail.com>

I have tried to google these issues , and found most of these papers ,
expect two ,
and i have update the list of  issue title , intro , and paper url in acm.

just get the list from attach file

-- krzycube

On 10/10/07, Ulf Wiger <ulf@REDACTED> wrote:
>
> The gproc paper and code can be found at
>
> svn co http://svn.ulf.wiger.net/gproc
>
> BR,
> Ulf W
>
> 2007/10/10, t ty <tty.erlang@REDACTED>:
> > The papers should be part of the ACM SIGPLAN library. Guess you have
> > to go thru them. Local university library is another good choice :)
> >
> > Perhaps the individual authors can make their papers available. I'm
> > not sure what the agreement with ACM is like.
> >
> > t
> >
> > On 10/9/07, krzycube <yuxh312@REDACTED> wrote:
> > > thx , tty
> > > and , where to download these papers?
> > >
> > > ---krzycube
> > >
> > >
> > > On 10/10/07, t ty <tty.erlang@REDACTED > wrote:
> > > >
> > > > It has been 4 days and Google still hasn't found any blog posts
> > > > regarding the workshop. You guys must be busy working :)
> > > >
> > > > Here is my quick write up, hopefully enough to interest folks to
> read
> > > > the papers and in attending EUC this year and the 2008 Erlang
> > > > Workshop. Unfortunately I could not stay for the discussion, perhaps
> > > > someone else could fill in the blanks.
> > > >
> > > > Disclaimer: omission and mistakes are my own.
> > > >
> > > > Regards
> > > >
> > > > t
> > > >
> > > > _______________________________________________
> > > > erlang-questions mailing list
> > > > erlang-questions@REDACTED
> > > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > > >
> > > >
> > >
> > >
> > >
> > > --
> > > "Yu Xihe" as Chinese Name
> > > "Martin" as English Name
> > > "KrzyCube" as Nickname
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>



-- 
"Yu Xihe" as Chinese Name
"Martin" as English Name
"KrzyCube" as Nickname
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/67d47762/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: erlangworkshop2007-issue-and-papers.txt
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/67d47762/attachment.txt>

From jcone@REDACTED  Wed Oct 10 08:14:11 2007
From: jcone@REDACTED (James Cone)
Date: Wed, 10 Oct 2007 19:14:11 +1300
Subject: [erlang-questions] Newbie: Is there a short-circuit for the comma
	operator?
Message-ID: <470C6DB3.8060700@eservglobal.com>

Hello All,

This isn't explicit in the Erlang Reference Manual (v5.5.5), so you may 
not think of it this way:

There is (something morally equivalent to) an operator ",", which is the 
lowest priority of all operators, and the evaluation of
    exp1, exp2
is:
   - have all of the side-effects of exp1
   - have all of the side-effects of exp2
   - return the value of exp2

eg

15> 0, 1.
1
16>


Is there an operator or function that short-circuits the comma operator, 
so if exp1 uses it, exp2 will not be evaluated?

eg (supposing it's called return, which is the name of the equivalent 
operator in C)

 > if 1==0 -> -1; true -> return 0 end, 1.
1
 >

I want this because I have a function with some deeply nested tests, and 
I want to tail-recurse from only some of the cases, without copying all 
of the following expressions (after a comma operator) into all of the 
other cases.

Am I making sense?  Does such a thing exist?

Regards,
James.


From yinso.chen@REDACTED  Wed Oct 10 08:39:11 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 9 Oct 2007 23:39:11 -0700
Subject: [erlang-questions] odbc: {port_exit,
	collecting_of_driver_information_failed}
Message-ID: <779bf2730710092339p578306d9pd4479766942987a8@mail.gmail.com>

Hi all -
I'm running into the following error when using odbc to connect against
postgresql.  I'm able to do the same with mysql, and I verified the odbc
connection working through isql & DataManager.

It seems that the connection was opened but then closed again, but I'm not
sure on how to proceed as my ODBC connection is working.  Any one else seen
this issue before?

Any pointers are appreciated, thanks.
yc

(emacs@REDACTED)102> {ok, Ref4} = odbc:connect("DSN=test3", []).
** exited: {{badmatch,{error,connection_closed}},[{erl_eval,expr,3}]} **

=ERROR REPORT==== 3-Oct-2007::19:12:11 ===
** Generic server <0.245.0> terminating
** Last message in was {#Port<0.244>,{exit_status,23}}
** When Server state == {state,#Port<0.244>,
                               {<0.243.0>,#Ref<0.0.0.2249>},
                               <0.243.0>,
                               undefined,
                               on,
                               undefined,
                               undefined,
                               on,
                               connecting,
                               undefined,
                               0,
                               [#Port<0.242>,#Port<0.243>],
                               #Port<0.245>,
                               #Port<0.246>}
** Reason for termination ==
** {port_exit,collecting_of_driver_information_faild}

=ERROR REPORT==== 3-Oct-2007::19:12:11 ===
Error in process <0.243.0> on node 'emacs@REDACTED' with exit value:
{{badmatch,{error,connection_closed}},[{erl_eval,expr,3}]}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071009/125045fd/attachment.htm>

From raimo+erlang-questions@REDACTED  Wed Oct 10 08:53:16 2007
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Wed, 10 Oct 2007 08:53:16 +0200
Subject: [erlang-questions] Newbie: Is there a short-circuit for the
	comma operator?
In-Reply-To: <470C6DB3.8060700@eservglobal.com>
References: <470C6DB3.8060700@eservglobal.com>
Message-ID: <20071010065316.GA28601@erix.ericsson.se>

Nope. Such a thing does not exist.

But it seems you want to do something that is well suited for throw/1.

try begin Exp1, Exp2, Exp3, Exp4 end of
   V -> V % Return value from Exp4
catch X ->
   X % Value from embedded throw/1
end

If any of ExpN evaluates throw(done),
the catch clause is evaluated with X bound to 'done'.

See also the Erts Reference manual.



On Wed, Oct 10, 2007 at 07:14:11PM +1300, James Cone wrote:
> Hello All,
> 
> This isn't explicit in the Erlang Reference Manual (v5.5.5), so you may 
> not think of it this way:
> 
> There is (something morally equivalent to) an operator ",", which is the 
> lowest priority of all operators, and the evaluation of
>     exp1, exp2
> is:
>    - have all of the side-effects of exp1
>    - have all of the side-effects of exp2
>    - return the value of exp2
> 
> eg
> 
> 15> 0, 1.
> 1
> 16>
> 
> 
> Is there an operator or function that short-circuits the comma operator, 
> so if exp1 uses it, exp2 will not be evaluated?
> 
> eg (supposing it's called return, which is the name of the equivalent 
> operator in C)
> 
>  > if 1==0 -> -1; true -> return 0 end, 1.
> 1
>  >
> 
> I want this because I have a function with some deeply nested tests, and 
> I want to tail-recurse from only some of the cases, without copying all 
> of the following expressions (after a comma operator) into all of the 
> other cases.
> 
> Am I making sense?  Does such a thing exist?
> 
> Regards,
> James.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From me@REDACTED  Wed Oct 10 08:54:07 2007
From: me@REDACTED (KatolaZ)
Date: Wed, 10 Oct 2007 08:54:07 +0200
Subject: [erlang-questions] Newbie: Is there a short-circuit for
	the	comma operator?
In-Reply-To: <470C6DB3.8060700@eservglobal.com>
References: <470C6DB3.8060700@eservglobal.com>
Message-ID: <20071010065407.GJ32134@katolaz.homeunix.net>

On Wed, Oct 10, 2007 at 07:14:11PM +1300, James Cone wrote:
> 
> Is there an operator or function that short-circuits the comma operator, 
> so if exp1 uses it, exp2 will not be evaluated?
> 
> eg (supposing it's called return, which is the name of the equivalent 
> operator in C)
> 
>  > if 1==0 -> -1; true -> return 0 end, 1.
> 1
>  >
> 
> I want this because I have a function with some deeply nested tests, and 
> I want to tail-recurse from only some of the cases, without copying all 
> of the following expressions (after a comma operator) into all of the 
> other cases.
> 
> Am I making sense?  Does such a thing exist?
> 

Such an operator does not exist in erlang. And I've never needed it
:-) You should perhaps rewrite your code better. Paste it, and let's
see where we can help.

HND

Enzo

-- 
[ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ]
[ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ]
[ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ]
[ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ]


From peppe@REDACTED  Wed Oct 10 09:07:51 2007
From: peppe@REDACTED (Peter Andersson)
Date: Wed, 10 Oct 2007 09:07:51 +0200
Subject: [erlang-questions] erl_errno on the Mac
In-Reply-To: <008107DA-E6D4-4E16-9A6B-F8A403886CF9@gmail.com>
References: <182DDFB1-1C56-4AB1-8E5A-F6309B2AB82F@gmail.com>
	<470BAF18.7030701@erix.ericsson.se>
	<008107DA-E6D4-4E16-9A6B-F8A403886CF9@gmail.com>
Message-ID: <470C7A47.40700@erix.ericsson.se>


Hi Joel,

I think ch 1.1. in the erl_interface user's guide answers your question:

"If you are using Erl_Interface functions in a threaded application
based on POSIX threads or Solaris threads, then Erl_Interface needs
access to some of the synchronization facilities in your threads
package, and you will need to specify additional compiler flags in order
to indicate which of the packages you are using. Define |_REENTRANT| and
either |STHREADS| or |PTHREADS|. The default is to use POSIX threads if
|_REENTRANT| is specified.

Note that both single threaded and default versions of the Erl_interface
and Ei libraries are provided. (The single threaded versions are named
|liberl_interface_st| and |libei_st|). Whether the default versions of
the libraries have support for threads or not is determined by if the
platform in question has support for POSIX or Solaris threads. To check
this, have a look in the |eidefs.mk| file in the erl_interface src
directory."

Peter


Joel Reymont wrote:
> Peter,
>
> On Oct 9, 2007, at 5:40 PM, Peter Andersson wrote:
>
> > #if !defined(_REENTRANT) && !defined(VXWORKS) && !defined(__WIN32__)
> > volatile int __erl_errno;
> > #endif
>
> I'm linking against libei.a, should I be using libei_st.a instead?
>
> 	Thanks, Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
>   



From jcone@REDACTED  Wed Oct 10 09:16:13 2007
From: jcone@REDACTED (James Cone)
Date: Wed, 10 Oct 2007 20:16:13 +1300
Subject: [erlang-questions] Newbie: Is there a short-circuit for
 the	comma operator?
In-Reply-To: <20071010065407.GJ32134@katolaz.homeunix.net>
References: <470C6DB3.8060700@eservglobal.com>
	<20071010065407.GJ32134@katolaz.homeunix.net>
Message-ID: <470C7C3D.2040802@eservglobal.com>

Hello KatolaZ and Raimo,

Raimo, I may have miscommunicated; I don't want the returned value; what 
I'm trying to achieve is to be tail-recursive from the middle of an if, 
where the non-recursing branches will evaluate another expression, after 
a comma-operator.  So throw/catch requires that I fail, because the 
stack frame will still be there, to catch the thrown thing.

KatolaZ, I'm trying to write the code below the way I would in C, where 
you control the nesting depth by making the control-flow leave the 
function as soon as possible.  It operates correctly in the short term, 
but is not tail-recursive, so (as expected) the interpreter runs out of 
memory.

Regards,
James.

KatolaZ wrote:
> On Wed, Oct 10, 2007 at 07:14:11PM +1300, James Cone wrote:
>> Is there an operator or function that short-circuits the comma operator, 
<snip>

> Such an operator does not exist in erlang. And I've never needed it
> :-) You should perhaps rewrite your code better. Paste it, and let's
> see where we can help.
<snip>

Raimo Niskanen wrote:
> Nope. Such a thing does not exist.
> 
> But it seems you want to do something that is well suited for throw/1.
<snip>

threadWatch(Socket, Label, Expect, Next, StatOn, StatTo, StatStart) ->
   % Expect and Next are serial numbers in our conversation with the server.
   % StatOn is the serial number on which to send the performance measure.
   % StatOn is normally false.  If it is -1, then the next send is the start
   % of the stats run.  If it is >= 0, then a stats run is in progress,
   % in which case StatStart (the start time) and StatTo (the stats process)
   % are defined.

   receive

     {tcp, error, _Reason} ->
       exit(_Reason);

     {tcp, Socket, Data} ->
       Colon = string:chr(Data, $:),
       if
         0 == Colon ->
           io:format("Unlabelled: ~p~n", [Data]),
           threadWatch(Socket, Label, Expect, Next, StatOn, StatTo, 
StatStart);
         true ->
           ok
       end,

       InLabel = string:substr(Data, 1, Colon-1),
       Rest = string:substr(Data, Colon+1),

%      io:format("Label = '~p', InLabel = '~p'~n", [Label, InLabel]),

       if
         InLabel == Label ->
           ok;
         true ->
           % This line is irrelevant; recurse now

%%%%
%%%% I believe the problem is here
%%%%
           threadWatch(Socket, Label, Expect, Next,
                              StatOn, StatTo, StatStart)
       end,

%%%
%%%  I want to avoid putting this code inside the
%%%  control structure above
%%%

       case erl_scan:string(Rest) of

         {ok, [{integer, 1, Got} |_], _} ->

%          io:format("Got = ~w, Expect = ~w~n", [Got, Expect]),

           if
             Expect == Got ->

               % This is where all of the important stuff happens.
               % Most of it is not implemented yet.

               Sendline = lists:concat([ Next,
" 
   \n"]),
               gen_tcp:send(Socket, Sendline),
%              io:format("Sent ~p", [Sendline]),
               threadWatch(Socket, Label, Expect+1, Next+1,
                           StatOn, StatTo, StatStart)
             ;
             true ->
               exit({badSequence, Data, Expect})
           end
         ;
         _ ->
           exit({malformedLine, Data})
       end

   end.


From hanssv@REDACTED  Wed Oct 10 10:11:23 2007
From: hanssv@REDACTED (Hans Svensson)
Date: Wed, 10 Oct 2007 10:11:23 +0200
Subject: [erlang-questions] Erlang Workshop 2007
In-Reply-To: <290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>	<e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com>
	<290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
Message-ID: <470C892B.2070502@cs.chalmers.se>

The pitfalls and recipes paper and also the paper on the more accurate 
semantics can be found at http://www.cs.chalmers.se/~hanssv/

I've also put the slides there in case anyone is interested!

Our Erlang related paper for ICFP is also there by the way ;-)

/Hans

t ty wrote:
> The papers should be part of the ACM SIGPLAN library. Guess you have
> to go thru them. Local university library is another good choice :)
> 
> Perhaps the individual authors can make their papers available. I'm
> not sure what the agreement with ACM is like.
> 
> t
> 
> On 10/9/07, krzycube <yuxh312@REDACTED> wrote:
>> thx , tty
>> and , where to download these papers?
>>
>> ---krzycube
>>
>>
>> On 10/10/07, t ty <tty.erlang@REDACTED > wrote:
>>> It has been 4 days and Google still hasn't found any blog posts
>>> regarding the workshop. You guys must be busy working :)
>>>
>>> Here is my quick write up, hopefully enough to interest folks to read
>>> the papers and in attending EUC this year and the 2008 Erlang
>>> Workshop. Unfortunately I could not stay for the discussion, perhaps
>>> someone else could fill in the blanks.
>>>
>>> Disclaimer: omission and mistakes are my own.
>>>
>>> Regards
>>>
>>> t
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>
>>
>> --
>> "Yu Xihe" as Chinese Name
>> "Martin" as English Name
>> "KrzyCube" as Nickname
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From raimo+erlang-questions@REDACTED  Wed Oct 10 10:42:54 2007
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Wed, 10 Oct 2007 10:42:54 +0200
Subject: [erlang-questions] : Newbie: Is there a short-circuit for
	the	comma operator?
In-Reply-To: <470C7C3D.2040802@eservglobal.com>
References: <470C6DB3.8060700@eservglobal.com>
	<20071010065407.GJ32134@katolaz.homeunix.net>
	<470C7C3D.2040802@eservglobal.com>
Message-ID: <20071010084254.GA136@erix.ericsson.se>

On Wed, Oct 10, 2007 at 08:16:13PM +1300, James Cone wrote:
> Hello KatolaZ and Raimo,
> 
> Raimo, I may have miscommunicated; I don't want the returned value; what 
> I'm trying to achieve is to be tail-recursive from the middle of an if, 
> where the non-recursing branches will evaluate another expression, after 
> a comma-operator.  So throw/catch requires that I fail, because the 
> stack frame will still be there, to catch the thrown thing.

In that case the standard solution is to restructure the code,
create a support function See below.


> 
> KatolaZ, I'm trying to write the code below the way I would in C, where 
> you control the nesting depth by making the control-flow leave the 
> function as soon as possible.  It operates correctly in the short term, 
> but is not tail-recursive, so (as expected) the interpreter runs out of 
> memory.
> 
> Regards,
> James.
> 
> KatolaZ wrote:
> > On Wed, Oct 10, 2007 at 07:14:11PM +1300, James Cone wrote:
> >> Is there an operator or function that short-circuits the comma operator, 
> <snip>
> 
> > Such an operator does not exist in erlang. And I've never needed it
> > :-) You should perhaps rewrite your code better. Paste it, and let's
> > see where we can help.
> <snip>
> 
> Raimo Niskanen wrote:
> > Nope. Such a thing does not exist.
> > 
> > But it seems you want to do something that is well suited for throw/1.
> <snip>
> 
> threadWatch(Socket, Label, Expect, Next, StatOn, StatTo, StatStart) ->
>    % Expect and Next are serial numbers in our conversation with the server.
>    % StatOn is the serial number on which to send the performance measure.
>    % StatOn is normally false.  If it is -1, then the next send is the start
>    % of the stats run.  If it is >= 0, then a stats run is in progress,
>    % in which case StatStart (the start time) and StatTo (the stats process)
>    % are defined.
> 
>    receive
> 
>      {tcp, error, _Reason} ->
>        exit(_Reason);
> 
>      {tcp, Socket, Data} ->
>        Colon = string:chr(Data, $:),
>        if
>          0 == Colon ->
>            io:format("Unlabelled: ~p~n", [Data]),
>            threadWatch(Socket, Label, Expect, Next, StatOn, StatTo, 
> StatStart);
>          true ->
>            ok
>        end,
> 
>        InLabel = string:substr(Data, 1, Colon-1),
>        Rest = string:substr(Data, Colon+1),
> 
> %      io:format("Label = '~p', InLabel = '~p'~n", [Label, InLabel]),
> 
>        if
>          InLabel == Label ->
	       threadWatch_1(Socket, Label, Expect, Next,
	                     StatOn, StatTo, StatStart, Data, Rest);
% >            ok;
>          true ->
>            % This line is irrelevant; recurse now
> 
> %%%%
> %%%% I believe the problem is here
> %%%%
>            threadWatch(Socket, Label, Expect, Next,
>                               StatOn, StatTo, StatStart)
%>        end,
         end
    end.
> 
> %%%
> %%%  I want to avoid putting this code inside the
> %%%  control structure above
> %%%
> 

threadWatch_1(Socket, Label, Expect, Next,
              StatOn, StatTo, StatStart, Data, Rest) ->
>        case erl_scan:string(Rest) of
> 
>          {ok, [{integer, 1, Got} |_], _} ->
> 
> %          io:format("Got = ~w, Expect = ~w~n", [Got, Expect]),
> 
>            if
>              Expect == Got ->
> 
>                % This is where all of the important stuff happens.
>                % Most of it is not implemented yet.
> 
>                Sendline = lists:concat([ Next,
> " 
>    \n"]),
>                gen_tcp:send(Socket, Sendline),
> %              io:format("Sent ~p", [Sendline]),
>                threadWatch(Socket, Label, Expect+1, Next+1,
>                            StatOn, StatTo, StatStart)
>              ;
>              true ->
>                exit({badSequence, Data, Expect})
>            end
>          ;
>          _ ->
>            exit({malformedLine, Data})
>        end
> 
>    end.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From S.J.Thompson@REDACTED  Wed Oct 10 11:01:46 2007
From: S.J.Thompson@REDACTED (S.J.Thompson)
Date: Wed, 10 Oct 2007 10:01:46 +0100 (BST)
Subject: [erlang-questions] Erlang Workshop 2007
Message-ID: <Pine.GSO.4.58.0710100957400.22677@myrtle.ukc.ac.uk>


Thanks to everyone who presented and attended the Erlang Workshop 2007 in
Freiburg last week. I should update you with what's happening next.

We hope to put up all the slides from the presentations in the next few
days, once we have received them all from the presenters; I will post once
this has happened. All the talks were also videoed, and these should be
available in due course.

The papers themselves are available in the ACM Digital Library, since this
is an ACM SIGPLAN sponsored workshop, but you may well find preliminary
versions of papers on contributors' websites.

Kind regards

Simon Thompson
Workshop Chair
Erlang Workshop 2007


From chandrashekhar.mullaparthi@REDACTED  Wed Oct 10 12:00:17 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 10 Oct 2007 11:00:17 +0100
Subject: [erlang-questions] Additional question regarding: gen_server vs
	gen_fsm
In-Reply-To: <2e23d1d20710091827n16f585f5t95985e1824205116@mail.gmail.com>
References: <2e23d1d20710091827n16f585f5t95985e1824205116@mail.gmail.com>
Message-ID: <b447a30d0710100300j3076ff2eue5ef12f8b1c8301b@mail.gmail.com>

Hi,

On 10/10/2007, David Budworth <dbudworth@REDACTED> wrote:
> Hi folks,
>
> YC's topic titled gen_server vs gen_fsm got me wondering
>
> Note for the reader: Sorry, just realized this is really long, but I'm
> really curious as to how/if this can be solved.  The last 3/4 of this is
> more bouncing ideas off people.
>
> I'm investigating writing an order routing system using erlang and can
> imagine a few different ways to deal with it (also an erlang newbie)
>
> in our case, the duration of a transaction can be months.
> So I'm assuming we really do need to persist state via mnesia (which I'll
> bug people about in upcoming messages, I'm sure).
>
> aside from that, there's a natural state progression for an order on an
> exchange, they tend to have states that go from:
> new -> confirmed -> partial_fill -> filled
> (and canceled, expired, etc depending on what really happens).
>
> At first glance of gen_fsm vs gen_server, it seems like gen_fsm is really a
> fancyish tracking mechanism built on top of (or really similar to)
> gen_server to keep state transition plumbing out of the 'business logic'
> area.
>
>
> so basically, the idea is (just looking for confirmation):
> Current State: confirmed
>
> confirmed(FillEvent,StateData) -> signal_done, {done,DoneStateData};
> confirmed(PartialFillEvent,StateData) ->
> {partial_fill,UpdatedStateData}.
>
> partial_fill(FillEvent,StateData) -> signal_done(),
> {done,DoneStateData}.
>
> something to that effect.
>
> so the question is, it seems like I'd have to manually persist "StateData"
> on each event handled to survive process/host failures and basically when
> detecting a process failure I'd have to spawn a new gen_fsm and register it
> (say with order id or something) passing in the most recently persisted
> state.
>
> Am I on the right track?

Yes, you are. You don't *have* to register it (I'm assuming you are
talking about process name registration.)

> I guess the only thing odd here, or rather, what I'm not entirely sure how
> it's handled is that with everything happening async in general, how does
> one guarantee that a message isn't lost while in transit and a host fails?
>
> coming from the JMS world, you receive a message in a TX, then do your work,
> publish a new message in that TX then commit.  If you die in the middle, the
> message is re-delivered to another listener sitting there.
>
> in the OTP world, what is the equivalent to this?
> meaning:
> SomeProcessThatSendsAnEvent:
>
> OrderHandlerPID ! MessageToProcess.
>
> OrderHandler
>    receive
>      Msg ->  start_doing_stuff(), crash().
>
> it seems like the only way to guarantee delivery is to make everything
> Request/Response synchronized.
> so you send a message then wait for an ack.  And if it doesn't come, you
> lookup/respawn/whatever a pid to re-send to until it succeeds.
>
> So if a process takes a long time, I'm blocking the sender.   Whereas in JMS
> (or JavaSpaces really) I know that if I send a message, that the message
> broker will Never(tm) lose the message and it will eventually be handled.
>
> Only way I see to make fast senders and reliable delivery would be to send a
> message to an intermediate process that persists/replicates to a buddy, and
> have the consumers send "Message Consume Requests" to that same intermediate
> process which will deliver the message and mark/delete it (still with a race
> condition since I may have deleted before sending or sent before deleting)
> giving me dups.
>
> it's almost as though what I really need is to use mnesia as my message bus
> so the "consumer" will do a select, process and submit an update.  This only
> works if mnesia supports operations similar to:
> begin tx
> insert (response values) in to msg table
> update (request row with DONE status) table where (request row) exists %% or
> delete the row, either way
> commit

mnesia does support this. And yes - to be really sure that every
message is handled, and only handled once, you have to use mnesia as
your message bus. You can have message senders insert into an mnesia
table and OrderHandler processes take one at a time and process them.
When you are done, you either change the state of the entry to DONE or
move it into another table in a single transaction.

> Is this type of thing just not what erlang is meant for?

Definitely. We do something exactly like this for our Third Party
Gateway. Messages from 3rd parties have to sent to mobile telephones
once and only once. We use mnesia for message queues. Each queue is
handled by a single process which spawns workers for each request in
the queue. The worker deletes the entry from the queue table when it
has successfully sent the message.

If a node restarts in the middle of queue processing, there might be
some entries in the database which have a status 'PROCESSING' but you
can't be sure whether the message was processed but the table did not
get updated, or whether the message did not get processed at all. Such
messages, we send failure reports to our 3rd parties which indicate
the uncertainity of that transaction, and delete them from the queue.

cheers
Chandru


From johanmon@REDACTED  Tue Oct  9 08:49:51 2007
From: johanmon@REDACTED (Johan Montelius)
Date: Tue, 09 Oct 2007 08:49:51 +0200
Subject: [erlang-questions] How random is this
In-Reply-To: <b447a30d0710082324n640b9103u965cf6650eb2d3cb@mail.gmail.com>
References: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>
	<b447a30d0710082324n640b9103u965cf6650eb2d3cb@mail.gmail.com>
Message-ID: <op.tzw3xdbxcuknpe@vuk.imit.kth.se>



I needed a procedure that made a process crash with random intervals so I  
wrote someting like this:

-module(crash).
-export([crash/0]).

crash() ->
   case random:uniform(100) of
	X when X < 10 ->
	  true = false;
	X ->
	  %% io:format("value ~w~n", [X]),
	  ok
   end.


works fine until it actually crashes, then the random function does not  
generate a new value, it's stuck in the value that caused the crash. Could  
be a feature, could be somthing else.

   Johan


-- 
Johan Montelius
KTH  ICT/ECS
There are 10 types of people in the world - those who understand binary  
and those who don't.


From ml.cyresse@REDACTED  Wed Oct 10 12:18:42 2007
From: ml.cyresse@REDACTED (Liam Clarke)
Date: Wed, 10 Oct 2007 23:18:42 +1300
Subject: [erlang-questions] regexp module
Message-ID: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>

Hi all,

Quick question, is there a way to do case insensitive matches with the
regexp module? I've hacked together a function to lower case
everything and go from there, but I just got that 'reinventing the
wheel' feeling that I get when in the early stages of familiarity with
a language and its libraries.

Regards,

Liam Clarke


From chandrashekhar.mullaparthi@REDACTED  Wed Oct 10 12:24:41 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 10 Oct 2007 11:24:41 +0100
Subject: [erlang-questions] How random is this
In-Reply-To: <op.tzw3xdbxcuknpe@vuk.imit.kth.se>
References: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>
	<b447a30d0710082324n640b9103u965cf6650eb2d3cb@mail.gmail.com>
	<op.tzw3xdbxcuknpe@vuk.imit.kth.se>
Message-ID: <b447a30d0710100324m5949386dmb91faef65537df6@mail.gmail.com>

On 09/10/2007, Johan Montelius <johanmon@REDACTED> wrote:
>
>
> I needed a procedure that made a process crash with random intervals so I
> wrote someting like this:
>
> -module(crash).
> -export([crash/0]).
>
> crash() ->
>    case random:uniform(100) of
>         X when X < 10 ->
>           true = false;
>         X ->
>           %% io:format("value ~w~n", [X]),
>           ok
>    end.
>
>
> works fine until it actually crashes, then the random function does not
> generate a new value, it's stuck in the value that caused the crash. Could
> be a feature, could be somthing else.
>

Try this instead.

-module(crash).
-export([crash/0]).

crash() ->
    {A, B, C} = now(),
    random:seed(A, B, C),
    crash_1().

crash_1() ->
  case random:uniform(100) of
       X when X < 10 ->
         io:format("value ~w~n", [X]),
         true = false;
       X ->
         crash_1()
  end.


From chsu79@REDACTED  Wed Oct 10 12:30:13 2007
From: chsu79@REDACTED (Christian S)
Date: Wed, 10 Oct 2007 12:30:13 +0200
Subject: [erlang-questions] How random is this
In-Reply-To: <op.tzw3xdbxcuknpe@vuk.imit.kth.se>
References: <DD923DB0-8E38-4216-9237-97D24DE07F3E@rodanotech.ch>
	<b447a30d0710082324n640b9103u965cf6650eb2d3cb@mail.gmail.com>
	<op.tzw3xdbxcuknpe@vuk.imit.kth.se>
Message-ID: <bc5c855a0710100330n45f93fcap6c2e544a559c0861@mail.gmail.com>

> I needed a procedure that made a process crash with random intervals so I
> wrote someting like this:

> works fine until it actually crashes, then the random function does not
> generate a new value, it's stuck in the value that caused the crash. Could
> be a feature, could be somthing else.

I don't really understand what problem you are having. But the random
module stores the current seed in the process dictionary, so each new
process starts off from the same fixed seed unless you change it
yourself.


> Johan Montelius
> KTH  ICT/ECS
> There are 10 types of people in the world - those who understand binary
> and those who don't.

Are you taking a KTH course that teaches Erlang or are you studying it
on your own?


From ml.cyresse@REDACTED  Wed Oct 10 14:04:34 2007
From: ml.cyresse@REDACTED (Liam Clarke)
Date: Thu, 11 Oct 2007 01:04:34 +1300
Subject: [erlang-questions] regexp module
In-Reply-To: <07E87BD707F5784F837C8381C7E8C67E013E74C3@esealmw111>
References: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
	<07E87BD707F5784F837C8381C7E8C67E013E74C3@esealmw111>
Message-ID: <b6f3249e0710100504i324f548eo32005477d24ad349@mail.gmail.com>

Ah, so there is, that'll teach me to not have the most up to date
documentation on my hard drive. (My docs are for stdlib 1.14.3)

Thanks Jonas.

On 10/11/07, Jonas ?man <jonas.aman@REDACTED> wrote:
> Hi,
>
> there is a function in string called to_lower. Other from that, I have no
> tips to give you. =/
>
> Regards,
> Jonas ?man
>
>
>
> -----Original Message-----
> From: erlang-questions-bounces@REDACTED
> [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Liam Clarke
> Sent: den 10 oktober 2007 12:19
> To: erlang-questions@REDACTED
> Subject: [erlang-questions] regexp module
>
> Hi all,
>
> Quick question, is there a way to do case insensitive matches with the
> regexp module? I've hacked together a function to lower case everything and
> go from there, but I just got that 'reinventing the wheel' feeling that I
> get when in the early stages of familiarity with a language and its
> libraries.
>
> Regards,
>
> Liam Clarke
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bjarne@REDACTED  Wed Oct 10 14:22:01 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Wed, 10 Oct 2007 14:22:01 +0200
Subject: [erlang-questions] EUC'2007 now open for registrations
Message-ID: <006801c80b38$29f8e270$671369d4@Dell>

All Erlang developers, researchers, users, programmers, and interested 
persons are hereby cordially invited to the 13th International Erlang/OTP 
User Conference, EUC, in Stockholm on November 8. Please see the Call for 
Papers at

http://www.erlang.se/euc/07/cfp.html

The dead-line is October 15.

If you wish to attend the EUC you have to register on before-hand. Please also indicate is you wish to participate in the planned ErLounge following the EUC.

Welcome

Bjarne D?cker
EUC chairman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/d1525ce4/attachment.htm>

From dbt@REDACTED  Wed Oct 10 16:39:42 2007
From: dbt@REDACTED (David Terrell)
Date: Wed, 10 Oct 2007 09:39:42 -0500
Subject: [erlang-questions] Stateful gen_servers
In-Reply-To: <C38F7BCE-AEA4-41FB-81D2-3E68BBE94EA5@ketralnis.com>
References: <317260.66052.qm@web81109.mail.mud.yahoo.com>
	<C38F7BCE-AEA4-41FB-81D2-3E68BBE94EA5@ketralnis.com>
Message-ID: <20071010143942.GK11915@sphinx.chicagopeoplez.org>

On Sat, Oct 06, 2007 at 06:45:17PM -0700, David King wrote:
> > I would like to use gen_server to develop a stateful server, and  
> > since gen_servers are  generally communicated using module:function 
> > (arg) syntax, what's the best way to use them as stateful servers  
> > per client, i.e., each for each client a gen_server is started and  
> > is terminated when client asks explicitly or when client is  
> > terminated. Thanks.
> 
> You could have each client do something like:
> 
> Session=my_server:logon(),
> Cookie=my_server:get_cookie(Session),
> my_server:eat_cookie(Session,Cookie), [...]
> 
> Then in the server, the State field could have a dict of Sessions  
> (generated via make_ref()?) to state information
> 
> If that's not enough, I have a tictactoe server that I wrote recently  
> that maintains the states of games this way, so let me know if you  
> want some live code that does this

What I've done for situations like this is have logon() call 
spawn_link(), without a registered name, and then do something
like:

get_cookie(Session) -> gen_server:call(Session, get_cookie).

Ideally the new gen_server would be added to a supervisor, not
spawned directly, but then call link() so that it shuts down
if the client pid exits.

-- 
David Terrell
dbt@REDACTED
((meatspace)) http://meat.net/


From dbt@REDACTED  Wed Oct 10 16:46:18 2007
From: dbt@REDACTED (David Terrell)
Date: Wed, 10 Oct 2007 09:46:18 -0500
Subject: [erlang-questions] gen_server vs gen_fsm
In-Reply-To: <779bf2730710081339o441847b8pdf7a91b3a1901a62@mail.gmail.com>
References: <779bf2730710080130q2ff8d787t8d57cfa6bdaccb05@mail.gmail.com>
	<b447a30d0710080152y27dfbed3g87a7393a7652376e@mail.gmail.com>
	<779bf2730710080213u6b993829se31abb9ff68d66a1@mail.gmail.com>
	<b447a30d0710080331x2c713611of3303f7a02086c95@mail.gmail.com>
	<779bf2730710081216x3cf999c0jae67bdc390b64a6f@mail.gmail.com>
	<8209f740710081316l3240b83akdd8cfb5488600706@mail.gmail.com>
	<779bf2730710081339o441847b8pdf7a91b3a1901a62@mail.gmail.com>
Message-ID: <20071010144618.GL11915@sphinx.chicagopeoplez.org>

On Mon, Oct 08, 2007 at 01:39:06PM -0700, YC wrote:
> Thanks.
> 
> Where can I find the definition of the State variable to figure out what's
> needed and not needed?

It is whatever you make it, so that would be up to you.

-- 
David Terrell
dbt@REDACTED
((meatspace)) http://meat.net/


From francesco@REDACTED  Wed Oct 10 16:58:05 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Wed, 10 Oct 2007 15:58:05 +0100
Subject: [erlang-questions] Erlang Workshop 2007: TCP/IP Connectivity
	Presentation
In-Reply-To: <Pine.GSO.4.58.0710100957400.22677@myrtle.ukc.ac.uk>
References: <Pine.GSO.4.58.0710100957400.22677@myrtle.ukc.ac.uk>
Message-ID: <470CE87D.5060609@erlang-consulting.com>

The paper / slides on Optimising TCP/IP connectivity in network 
intensive Erlang systems are already online.

Paper: 
http://www.erlang-consulting.com/thesis/tcp_optimisation/tcp_optimisation.html 

Slides: 
http://www.erlang-consulting.com/thesis/tcp_optimisation/tcp_optimisation_slides.pdf

*Abstract:* With the increased Use of network enabled applications and 
server hosted software systems, scalability with respect to network 
connectivity is becoming an increasingly important subject. The 
programming language Erlang has previously been shown to be a suitable 
choice for creating highly available, scalable and robust telecoms 
systems. In this exploratory study we want to investigate how to 
optimise an Erlang system for maximum TCP/IP connectivity in terms of 
operating system, tuning of the operating system TCP stack and tuning of 
the Erlang Runtime System. The study shows how a series of benchmarks 
are used to evaluate the impact of these factors and how to evaluate the 
best settings for deploying and configuring an Erlang application. We 
conclude that the choice of operating system and the use of kernel poll 
both have a major impact on the scalability of the benchmarked systems.

The paper and slides on priority messaging and the presentation on 
comparing Erlang and C++ from CUFP will be put online tomorrow.

Regards,
Francesco
--
http://www.erlang-consulting.com





S.J.Thompson wrote:
> Thanks to everyone who presented and attended the Erlang Workshop 2007 in
> Freiburg last week. I should update you with what's happening next.
>
> We hope to put up all the slides from the presentations in the next few
> days, once we have received them all from the presenters; I will post once
> this has happened. All the talks were also videoed, and these should be
> available in due course.
>
> The papers themselves are available in the ACM Digital Library, since this
> is an ACM SIGPLAN sponsored workshop, but you may well find preliminary
> versions of papers on contributors' websites.
>
> Kind regards
>
> Simon Thompson
> Workshop Chair
> Erlang Workshop 2007
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>
>   



From Tim.Bray@REDACTED  Wed Oct 10 19:57:13 2007
From: Tim.Bray@REDACTED (Tim Bray)
Date: Wed, 10 Oct 2007 10:57:13 -0700
Subject: [erlang-questions] regexp module
In-Reply-To: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
References: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
Message-ID: <5D97B0A4-7058-468A-957D-6EBACB85B39C@sun.com>


On Oct 10, 2007, at 3:18 AM, Liam Clarke wrote:

> Hi all,
>
> Quick question, is there a way to do case insensitive matches with the
> regexp module? I've hacked together a function to lower case
> everything and go from there, but I just got that 'reinventing the
> wheel' feeling that I get when in the early stages of familiarity with
> a language and its libraries.

If you're going to have to handle internationalized text in the  
general case, it's probably better to stay away from case-folding.   
The rules are incredibly locale-sensitive and language-sensitive.   
The java.String.toLower() call is insanely slow because it tries to  
deal with all these corner cases.  If you're in ASCII, you're OK, but  
even ISO-Latin-1 gets into trouble?  For example, what's the lower- 
case of "I"?  It's different in Turkey.  -Tim




From rvirding@REDACTED  Wed Oct 10 21:29:44 2007
From: rvirding@REDACTED (Robert Virding)
Date: Wed, 10 Oct 2007 21:29:44 +0200
Subject: [erlang-questions] Erlang AST with locations
In-Reply-To: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
References: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
Message-ID: <3dbc6d1c0710101229j6759ebbdi12b6acdd674f8746@mail.gmail.com>

On 05/10/2007, Joel Reymont <joelr1@REDACTED> wrote:
>
> The result of erl_scan:string/1 has line numbers in it but these get
> lost once erl_parse:parse/1 runs.
>
> I would like to have an AST with locations, is there a way to
> accomplish that?


Sorry I don't really understand what you mean. If you do

28> {ok,Ts,_}=erl_scan:string("abc([a])->\n1.").
{ok,[{atom,1,abc},
     {'(',1},
     {'[',1},
     {atom,1,a},
     {']',1},
     {')',1},
     {'->',1},
     {integer,2,1},
     {dot,2}],
    2}
29> erl_parse:parse(Ts).
{ok,{function,1,
              abc,
              1,

[{clause,1,[{cons,1,{atom,1,a},{nil,1}}],[],[{integer,2,1}]}]}}

you that you do have the line numbers in the AST. If you use parse_term,
however, line numbers are removed as it is the term which returned not the
AST.

On a related note, I think it's a bummer that there's no
> erl_scan:binary/1.


True, true. One of these years ...

Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071010/70707a6b/attachment.htm>

From kenneth.lundin@REDACTED  Wed Oct 10 22:52:54 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Wed, 10 Oct 2007 22:52:54 +0200
Subject: [erlang-questions] Erlang AST with locations
In-Reply-To: <3dbc6d1c0710101229j6759ebbdi12b6acdd674f8746@mail.gmail.com>
References: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
	<3dbc6d1c0710101229j6759ebbdi12b6acdd674f8746@mail.gmail.com>
Message-ID: <de422a40710101352q4c90e532oe33dc904e55e5660@mail.gmail.com>

On 10/10/07, Robert Virding <rvirding@REDACTED> wrote:
> On 05/10/2007, Joel Reymont <joelr1@REDACTED> wrote:
> > The result of erl_scan:string/1 has line numbers in it but these get
> > lost once erl_parse:parse/1 runs.
> >
> > I would like to have an AST with locations, is there a way to
> > accomplish that?
>
> Sorry I don't really understand what you mean. If you do
>
For example from erl_scan:string below you want the result (in
principle,but maybe
somewhat optimized and compressed)
{ok,[{atom,Location1,abc},
>      {'(',Location2},
>      {'[',Location3},
>      {atom,Location3,a},
>      {']',Location4},
>      {')',Location5},
>      {'->',Location6},
>      {integer,Location7,1},
>      {dot,Location8}],
>     2}

where Location could be {Line,Col,SourceText}
or {FilePos,Length} or {Line,Col,FilePos,OriginalText} to be as
flexible as possible
and for each token. But of course OriginalText is not necessary for all tokens


> 29> erl_parse:parse(Ts).
> {ok,{function,1,
>               abc,
>               1,
>
> [{clause,1,[{cons,1,{atom,Location1,a},{nil,Location2}}],[],[{integer,Location3,1}]}]}}
>
> you that you do have the line numbers in the AST. If you use parse_term,
> however, line numbers are removed as it is the term which returned not the
> AST.
> > On a related note, I think it's a bummer that there's no
> > erl_scan:binary/1.

I don't agree
Why should it be important to have a erl_scan:binary?
When it is so damn easy to do:
{ok,Bin} = file:read_file(File),
Str = binary_to_list(Bin),
erl_scan:string(Str),

Why should we double the same code for a list of characters and a
binary of bytes?
when we easily can do as above.

Of course we can provide a erl_scan:binary/1 which works as above
if that makes enough amount of people happy.

I think we should have a scanner which supports "Locations" in the
original distribution so that Editor and IDE projects as ErlIde don't
need to invent their
own special scanner.
It will happen quite soon but I can't make any promises regarding when.


/Kenneth (Erlang/OTP group at Ericsson)


From erlangx@REDACTED  Wed Oct 10 23:33:01 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Wed, 10 Oct 2007 14:33:01 -0700
Subject: [erlang-questions] regexp module
In-Reply-To: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
References: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
Message-ID: <20071010213301.GZ6096@delora.autosys.us>

On Wed, Oct 10, 2007 at 11:18:42PM +1300, Liam Clarke wrote:
> Hi all,
> 
> Quick question, is there a way to do case insensitive matches with the
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A bit of a pain, but you could do ...

1> regexp:matches("This IS the string, isn't it?", "[Ii][Ss]").
{match,[{3,2},{6,2},{21,2}]}
2>

I force some fields to all upper case myself for fixed length user
input data.

~Michael

> regexp module? I've hacked together a function to lower case
> everything and go from there, but I just got that 'reinventing the
> wheel' feeling that I get when in the early stages of familiarity with
> a language and its libraries.
> 
> Regards,
> 
> Liam Clarke
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,470ca71773321088629368!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From vladdu55@REDACTED  Wed Oct 10 23:39:57 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Wed, 10 Oct 2007 21:39:57 +0000
Subject: [erlang-questions] Erlang AST with locations
In-Reply-To: <de422a40710101352q4c90e532oe33dc904e55e5660@mail.gmail.com>
References: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
	<3dbc6d1c0710101229j6759ebbdi12b6acdd674f8746@mail.gmail.com>
	<de422a40710101352q4c90e532oe33dc904e55e5660@mail.gmail.com>
Message-ID: <95be1d3b0710101439s3edbfd72q348cfa7aa0f93eba@mail.gmail.com>

Hi,

On 10/10/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
> I think we should have a scanner which supports "Locations" in the
> original distribution so that Editor and IDE projects as ErlIde don't
> need to invent their
> own special scanner.
> It will happen quite soon but I can't make any promises regarding when.

This is great!

As a side note, but related, it should be remembered that the scanner
and especially the parser that is needed by a code processing tool
(like Erlide) has somewhat different requirements from when used
inside a compiler. Namely, in an IDE they should reflect the code as
it is written as opposed to its meaning.

So for example, in an IDE one also needs whitespace and comments, but
they have to be easy to discard. Preprocessing directives are also
significant in an IDE, while the "real" parser doesn't ever sees
those. Furthermore, in an IDE these tools have to be fault tolerant
(code in the process of being written shouldn't cause errors) and
incremental.

How to fulfill the requirements of both kinds of tools in the same
scanner/parser is a quite difficult question. I think it is possible
for the scanner, but not for the parser. The future will tell.

best regards,
Vlad


From jao@REDACTED  Thu Oct 11 04:00:02 2007
From: jao@REDACTED (Jack Orenstein)
Date: Wed, 10 Oct 2007 22:00:02 -0400
Subject: [erlang-questions] Difference between gen_server and gen_fsm
Message-ID: <E14B0D03-25E6-43F9-A6A0-4D50508613D0@geophile.com>

I'm going through the OTP design principles book and think that I am  
missing the point of gen_fsm. The difference between gen_fsm and  
gen_server seems to be a small syntactic one. Instead of handle_call  
handling each kind of request, there is a function per FSM state. Not  
having used gen_fsm yet, it seems that gen_fsm would lead to cleaner  
code. But the two behaviors are so similar, I feel that I am either  
missing the point of gen_fsm, or completely missing some aspect of  
its behavior. Can someone please explain the distinction?

Jack Orenstein



From jcone@REDACTED  Thu Oct 11 08:05:21 2007
From: jcone@REDACTED (James Cone)
Date: Thu, 11 Oct 2007 19:05:21 +1300
Subject: [erlang-questions] Newbie: Is there a short-circuit for
	the	comma operator?
In-Reply-To: <mailman.1.1192010401.2772.erlang-questions@erlang.org>
References: <mailman.1.1192010401.2772.erlang-questions@erlang.org>
Message-ID: <470DBD21.7050508@eservglobal.com>

Thanks; I like auxilliary functions as a general technique.  The only 
way it doesn't scale is the shopping-list of command-line arguments, and 
I guess the answer to that is records.

The code also gets simpler when I start using erlang:list_to_integer. 
Should something like that be in the getting-started manual, near 
io:format()?

>> Raimo, I may have miscommunicated; I don't want the returned value; what 
>> > I'm trying to achieve is to be tail-recursive from the middle of an if, 
>> > where the non-recursing branches will evaluate another expression, after 
>> > a comma-operator.  So throw/catch requires that I fail, because the 
>> > stack frame will still be there, to catch the thrown thing.
> 
> In that case the standard solution is to restructure the code,
> create a support function See below.



From ulf@REDACTED  Thu Oct 11 08:39:18 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Thu, 11 Oct 2007 08:39:18 +0200
Subject: [erlang-questions] Difference between gen_server and gen_fsm
In-Reply-To: <E14B0D03-25E6-43F9-A6A0-4D50508613D0@geophile.com>
References: <E14B0D03-25E6-43F9-A6A0-4D50508613D0@geophile.com>
Message-ID: <8209f740710102339y384d2b62lb7ac07a90b617cf3@mail.gmail.com>

I agree - gen_fsm can lead to cleaner code, and this might be
significant esp in state machines with many states.

BR,
Ulf W

2007/10/11, Jack Orenstein <jao@REDACTED>:
> I'm going through the OTP design principles book and think that I am
> missing the point of gen_fsm. The difference between gen_fsm and
> gen_server seems to be a small syntactic one. Instead of handle_call
> handling each kind of request, there is a function per FSM state. Not
> having used gen_fsm yet, it seems that gen_fsm would lead to cleaner
> code. But the two behaviors are so similar, I feel that I am either
> missing the point of gen_fsm, or completely missing some aspect of
> its behavior. Can someone please explain the distinction?
>
> Jack Orenstein
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From raimo+erlang-questions@REDACTED  Thu Oct 11 09:24:00 2007
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Thu, 11 Oct 2007 09:24:00 +0200
Subject: [erlang-questions] : Newbie: Is there a short-circuit for
	the	comma operator?
In-Reply-To: <470DBD21.7050508@eservglobal.com>
References: <mailman.1.1192010401.2772.erlang-questions@erlang.org>
	<470DBD21.7050508@eservglobal.com>
Message-ID: <20071011072400.GA11628@erix.ericsson.se>

On Thu, Oct 11, 2007 at 07:05:21PM +1300, James Cone wrote:
> Thanks; I like auxilliary functions as a general technique.  The only 
> way it doesn't scale is the shopping-list of command-line arguments, and 
> I guess the answer to that is records.

Shopping-list?

English is not my native language, so I assume you mean the number of
function arguments tends to grow to painful numbers. In that case:
yes, records can help.

But if you get a very long list of function arguments you should
think again. Perhaps the cut between the functions can be made
at a different place. Fewer arguments tends to give more
maintainable code since the programmer gets a smaller
state to keep in the head when understanding the code.

Somtimes helper functions like the one I suggested take a
lot of arguments just to be able to do the tail recursion,
but do not use them themselves. In that case there might
be a different function structure with a few arguments to the
helper function, and a few (or one) return value that indicate
if tail recursion should be done, and the calling
function does the tail recursion.



> 
> The code also gets simpler when I start using erlang:list_to_integer. 
> Should something like that be in the getting-started manual, near 
> io:format()?
> 
> >> Raimo, I may have miscommunicated; I don't want the returned value; what 
> >> > I'm trying to achieve is to be tail-recursive from the middle of an if, 
> >> > where the non-recursing branches will evaluate another expression, after 
> >> > a comma-operator.  So throw/catch requires that I fail, because the 
> >> > stack frame will still be there, to catch the thrown thing.
> > 
> > In that case the standard solution is to restructure the code,
> > create a support function See below.
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From yerl@REDACTED  Thu Oct 11 09:25:42 2007
From: yerl@REDACTED (yerl@REDACTED)
Date: Thu, 11 Oct 2007 09:25:42 +0200
Subject: [erlang-questions] Encrypt beam
Message-ID: <mnet1.1192087542.19764.yerl@club-internet.fr>

An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071011/4d159a5c/attachment.htm>

From bjorn@REDACTED  Thu Oct 11 09:27:10 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 11 Oct 2007 09:27:10 +0200
Subject: [erlang-questions] Erlang AST with locations
In-Reply-To: <de422a40710101352q4c90e532oe33dc904e55e5660@mail.gmail.com>
References: <562CC284-A8C1-47C9-AEAE-0781665BF987@gmail.com>
	<3dbc6d1c0710101229j6759ebbdi12b6acdd674f8746@mail.gmail.com>
	<de422a40710101352q4c90e532oe33dc904e55e5660@mail.gmail.com>
Message-ID: <o1wstuaz75.fsf@erix.ericsson.se>

"Kenneth Lundin" <kenneth.lundin@REDACTED> writes:

> 
> I don't agree
> Why should it be important to have a erl_scan:binary?
> When it is so damn easy to do:
> {ok,Bin} = file:read_file(File),
> Str = binary_to_list(Bin),
> erl_scan:string(Str),

One reason is that the list will need 8 times as much memory (in 32-bit emulator)
as the binary.

/Bjorn 
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From rsaccon@REDACTED  Thu Oct 11 10:17:43 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 11 Oct 2007 05:17:43 -0300
Subject: [erlang-questions] Encrypt beam
In-Reply-To: <mnet1.1192087542.19764.yerl@club-internet.fr>
References: <mnet1.1192087542.19764.yerl@club-internet.fr>
Message-ID: <eb4177b00710110117s17892a62xfda0dee3e51b1230@mail.gmail.com>

take a look at the beam_lib documentation

http://erlang.org/doc/man/beam_lib.html


On 10/11/07, yerl@REDACTED <yerl@REDACTED> wrote:
>
>  Hi guys !
>
> I'm looking for a simple to encrypt ".beam" files to be sure that nobody
> could reverse engineer them and got the original source (.erl).
>
> For now, I'm just removing the debugging stuffs from them (.beam). Is it
> enough for my problem ?
>
> Thanks in advance
>
> cheers
> Y.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Roberto Saccon
http://rsaccon.com


From findraju@REDACTED  Thu Oct 11 11:04:35 2007
From: findraju@REDACTED (Raj)
Date: Thu, 11 Oct 2007 14:34:35 +0530
Subject: [erlang-questions] Crash in erlang , inet_dns:dns_exp
Message-ID: <57e317240710110204v7b629b62y2bd4c3ca0aee7578@mail.gmail.com>

Hi all,

 I got a crash in erlang  when I tried to send DNS queries continuously.
What I could understand is that, inet_dns:dns_exp is not handling the
message properly. I am pasting the log below.
Please comment on this.


** Reason for termination =

** {function_clause,[{inet_dns,dn_exp,

[[100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[254,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[]]},

{inet_dns,decode_query_section,4},

{inet_dns,decode_sections,6},

{inet_dns,decode,1},

{dns_query,decode,2},

{gen_fsm,handle_msg,7},

{proc_lib,init_p,5}]}

== 2-Mar-2007::15:58:09 == ERROR REPORT - crash_report <0.25103.0>

[[{pid,<0.25103.0>},

{registered_name,[]},

{error_info,{function_clause,[{inet_dns,

dn_exp,

[[100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[254,

0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

[]]},

{inet_dns,decode_query_section,4},

{inet_dns,decode_sections,6},

{inet_dns,decode,1},

{dns_query,decode,2},

{gen_fsm,handle_msg,7},

{proc_lib,init_p,5}]}},

{initial_call,{gen,init_it,

[gen_fsm,

<0.481.0>,

<0.481.0>,

dns_query,

[[254,

0,

0,

0|

<<1,0,0,0,0,0,0,7,100,110,115,116,111,111,108,11,99,111,100,101,110,111,109,105,99,111,110,5,108,111,99,97,108,0,0,1,0,1>>],

{10,56,210,2},

53,

udp,

{options,false,false,false,false,false},

{10,56,40,100},

<0.481.0>],

[]]}},

{ancestors,[<0.481.0>,dns_server,dns_sup,<0.469.0>]},

{messages,[]},

{links,[<0.481.0>]},

{dictionary,[]},

{trap_exit,true},

{status,running},

{heap_size,987},

{stack_size,21},

{reductions,214}],

[]]

== 2-Mar-2007::15:58:10 == ERROR - <0.25105.0>

** State machine <0.25105.0> terminating

** Last event in was timeout

** When State == decode

** Data == {state,[0,

0,

0,

1,

0,

0,

0,

0,

0,

0,

7,

100,

110,

115,

116,

111,

111,

108,

11,

99,

111,

100,

101,

110,

111,

109,

105,

99,

111,

110,

5,

108,

111,

99,

97,

108,

0,

0,

1,

0,

1],

{{10,56,210,2},53,udp},

undefined,

undefined,

undefined,

undefined,

undefined,

0,

undefined,

undefined,

{options,false,false,false,false,false},

{10,56,40,100},

<0.481.0>}

Any help would be appreciated

Thanks,

Raju
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071011/06c39ae8/attachment.htm>

From ulf.wiger@REDACTED  Thu Oct 11 11:09:50 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Thu, 11 Oct 2007 11:09:50 +0200
Subject: [erlang-questions] Encrypt beam
In-Reply-To: <eb4177b00710110117s17892a62xfda0dee3e51b1230@mail.gmail.com>
References: <mnet1.1192087542.19764.yerl@club-internet.fr>
	<eb4177b00710110117s17892a62xfda0dee3e51b1230@mail.gmail.com>
Message-ID: <470DE85E.9080502@ericsson.com>

Roberto Saccon wrote:
> take a look at the beam_lib documentation
> 
> http://erlang.org/doc/man/beam_lib.html
> 

To clarify, beam_lib offers a way to encrypt the
'debug_info' (the source in parsed form) which can be
embedded in the beam file. This allows you to embed
debug_info even if you don't want to reveal the source
code. The encryption can be unlocked at runtime, so that
you can e.g. run profiling, etc. (perhaps eventually even
pretty-print the source and view it through distel?)

The beam code itself can of course be reverse-engineered.
The format isn't documented, but to get some idea of
what can possibly be extracted from beam code, you can view
the ASM code, from which the beam code is produced. How to
do this _is_ documented. Whether this is acceptable
obfuscation of your code, is of course up to you.

See an example at the end of this mail.

BR,
Ulf W


> 
> On 10/11/07, yerl@REDACTED <yerl@REDACTED> wrote:
>>  Hi guys !
>>
>> I'm looking for a simple to encrypt ".beam" files to be sure that nobody
>> could reverse engineer them and got the original source (.erl).
>>
>> For now, I'm just removing the debugging stuffs from them (.beam). Is it
>> enough for my problem ?
>>
>> Thanks in advance
>>
>> cheers
>> Y.
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions

= = = = = Example:

***** lc.erl:

-module(lc).

-export([lc1/1,
          lc2/2]).


lc1(List) ->
     L = [foo(X) ||
             X <- List,
             X > 17],
     lists:reverse(L).

lc2(List1, List2) ->
     [foo({X, Y}) ||
         X <- List1,
         Y <- List2,
         X > Y].

foo(X) -> X.

*****
erlc -W -S lc.erl

***** lc.S:

{module, lc}.  %% version = 0

{exports, [{lc1,1},{lc2,2},{module_info,0},{module_info,1}]}.

{attributes, []}.

{labels, 22}.


{function, lc1, 1, 2}.
   {label,1}.
     {func_info,{atom,lc},{atom,lc1},1}.
   {label,2}.
     {allocate,0,1}.
     {'%live',1}.
     {call,1,{f,19}}.
     {call_ext_last,1,{extfunc,lists,reverse,1},0}.


{function, lc2, 2, 4}.
   {label,3}.
     {func_info,{atom,lc},{atom,lc2},2}.
   {label,4}.
     {call_only,2,{f,12}}.


{function, foo, 1, 6}.
   {label,5}.
     {func_info,{atom,lc},{atom,foo},1}.
   {label,6}.
     return.


{function, module_info, 0, 8}.
   {label,7}.
     {func_info,{atom,lc},{atom,module_info},0}.
   {label,8}.
     {move,{atom,lc},{x,0}}.
     {call_ext_only,1,{extfunc,erlang,get_module_info,1}}.


{function, module_info, 1, 10}.
   {label,9}.
     {func_info,{atom,lc},{atom,module_info},1}.
   {label,10}.
     {move,{x,0},{x,1}}.
     {move,{atom,lc},{x,0}}.
     {call_ext_only,2,{extfunc,erlang,get_module_info,2}}.


{function, '-lc2/2-lc$^0/1-0-', 2, 12}.
   {label,11}.
     {func_info,{atom,lc},{atom,'-lc2/2-lc$^0/1-0-'},2}.
   {label,12}.
     {test,is_nonempty_list,{f,13},[{x,0}]}.
     {get_list,{x,0},{x,3},{x,2}}.
     {move,{x,1},{x,0}}.
     {call_only,4,{f,15}}.
   {label,13}.
     {test,is_nil,{f,11},[{x,0}]}.
     return.


{function, '-lc2/2-lc$^1/1-1-', 4, 15}.
   {label,14}.
     {func_info,{atom,lc},{atom,'-lc2/2-lc$^1/1-1-'},4}.
   {label,15}.
     {test,is_nonempty_list,{f,17},[{x,0}]}.
     {get_list,{x,0},{x,4},{x,5}}.
     {test,is_lt,{f,16},[{x,4},{x,3}]}.
     {allocate_heap,4,3,6}.
     {move,{x,1},{y,3}}.
     {move,{x,2},{y,2}}.
     {move,{x,3},{y,1}}.
     {move,{x,5},{y,0}}.
     {put_tuple,2,{x,0}}.
     {put,{x,3}}.
     {put,{x,4}}.
     {'%live',1}.
     {call,1,{f,6}}.
     {move,{x,0},{x,4}}.
     {move,{y,2},{x,2}}.
     {move,{y,3},{x,1}}.
     {move,{y,1},{x,3}}.
     {move,{y,0},{x,0}}.
     {move,{x,4},{y,3}}.
     {'%live',4}.
     {kill,{y,0}}.
     {kill,{y,1}}.
     {kill,{y,2}}.
     {call,4,{f,15}}.
     {test_heap,2,1}.
     {put_list,{y,3},{x,0},{x,0}}.
     {'%live',1}.
     {deallocate,4}.
     return.
   {label,16}.
     {move,{x,5},{x,0}}.
     {call_only,4,{f,15}}.
   {label,17}.
     {test,is_nil,{f,14},[{x,0}]}.
     {move,{x,2},{x,0}}.
     {call_only,2,{f,12}}.


{function, '-lc1/1-lc$^0/1-0-', 1, 19}.
   {label,18}.
     {func_info,{atom,lc},{atom,'-lc1/1-lc$^0/1-0-'},1}.
   {label,19}.
     {test,is_nonempty_list,{f,21},[{x,0}]}.
     {get_list,{x,0},{x,1},{x,2}}.
     {test,is_lt,{f,20},[{integer,17},{x,1}]}.
     {allocate,1,3}.
     {move,{x,1},{x,0}}.
     {move,{x,2},{y,0}}.
     {'%live',1}.
     {call,1,{f,6}}.
     {move,{x,0},{x,1}}.
     {move,{y,0},{x,0}}.
     {move,{x,1},{y,0}}.
     {'%live',1}.
     {call,1,{f,19}}.
     {test_heap,2,1}.
     {put_list,{y,0},{x,0},{x,0}}.
     {'%live',1}.
     {deallocate,1}.
     return.
   {label,20}.
     {move,{x,2},{x,0}}.
     {call_only,1,{f,19}}.
   {label,21}.
     {test,is_nil,{f,18},[{x,0}]}.
     return.


From ml.cyresse@REDACTED  Thu Oct 11 11:25:05 2007
From: ml.cyresse@REDACTED (Liam Clarke)
Date: Thu, 11 Oct 2007 22:25:05 +1300
Subject: [erlang-questions] regexp module
In-Reply-To: <5D97B0A4-7058-468A-957D-6EBACB85B39C@sun.com>
References: <b6f3249e0710100318v20ee78adra4f7eca1b35e2e32@mail.gmail.com>
	<5D97B0A4-7058-468A-957D-6EBACB85B39C@sun.com>
Message-ID: <b6f3249e0710110225q1517e803j30e92d260bfa59f1@mail.gmail.com>

On 10/11/07, Tim Bray <Tim.Bray@REDACTED> wrote:
>
> On Oct 10, 2007, at 3:18 AM, Liam Clarke wrote:
>
> > Hi all,
> >
> > Quick question, is there a way to do case insensitive matches with the
> > regexp module? I've hacked together a function to lower case
> > everything and go from there, but I just got that 'reinventing the
> > wheel' feeling that I get when in the early stages of familiarity with
> > a language and its libraries.
>
> If you're going to have to handle internationalized text in the
> general case, it's probably better to stay away from case-folding.
> The rules are incredibly locale-sensitive and language-sensitive.
> The java.String.toLower() call is insanely slow because it tries to
> deal with all these corner cases.  If you're in ASCII, you're OK, but
> even ISO-Latin-1 gets into trouble?  For example, what's the lower-
> case of "I"?  It's different in Turkey.  -Tim

Thanks for the heads up, I'll bear that in mind. it's HTTP headers, so
i18n isn't a concern at this point. Unless they've snuck something
sneaky into the HTTP spec. >_<

>A bit of a pain, but you could do ...
>
>1> regexp:matches("This IS the string, isn't it?", "[Ii][Ss]").
>{match,[{3,2},{6,2},{21,2}]}
>2>
>
>I force some fields to all upper case myself for fixed length user
>input data.
>
>~Michael

Oh yeah. /me smacks forehead. I could do it like that. I was all hung
up on having some kind of IGNORECASE flag.

Thanks,

Liam


From cthulahoops@REDACTED  Thu Oct 11 12:49:36 2007
From: cthulahoops@REDACTED (Adam Kelly)
Date: Thu, 11 Oct 2007 11:49:36 +0100
Subject: [erlang-questions] Newbie: Is there a short-circuit for the
	comma operator?
In-Reply-To: <470DBD21.7050508@eservglobal.com>
References: <mailman.1.1192010401.2772.erlang-questions@erlang.org>
	<470DBD21.7050508@eservglobal.com>
Message-ID: <8d1798e90710110349x6bad5a0fmbec27db5bcad08b1@mail.gmail.com>

> Thanks; I like auxilliary functions as a general technique.  The only
> way it doesn't scale is the shopping-list of command-line arguments, and
> I guess the answer to that is records.

Rather than creating an auxilary function with a long argument list,
looking at splitting each task into its own function.  That one long
function should be four or five:

Receive a message.
Split a line of text at a colon.
Parse the data to an integer.
Format and send the data.

> The code also gets simpler when I start using erlang:list_to_integer.

Yes, but whenever you find yourself wanting a missing library
function, implement it!

list_to_integer(X) ->
  case erl_scan:string(X) of
     {ok,[{integer,1,Y}],1} -> Y;
     _ -> throw(badarg)
  end.

That way your code stays simple, and you can easily change to the
library function when you find it!

Adam.


From francesco@REDACTED  Thu Oct 11 13:09:40 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Thu, 11 Oct 2007 12:09:40 +0100
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
 Conference going to be?
In-Reply-To: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
Message-ID: <470E0474.8020104@erlang-consulting.com>

Hi Garry,

I agree, the Erlang workshop was the best one so far. 100 participants 
at the Commercial Users of Functional Programming (Growing at this rate, 
the workshop will become larger than ICFP ;-) ) and 60 at the Erlang 
workshop are real records clearly showing the positive trends in the 
industrial use of FP.

One of the things we have done for the past three EUCs is provide Erlang 
and OTP courses in conjunction with the Conference. The week before the 
conference, we give a 5 day Erlang by Example, and on the week of teh 
conference, a 4 day OTP course, with a one day break on the Thursday for 
the conference itself. More information on the courses is available at

http://www.erlang-consulting.com/training/schedule.html

See you at EUC!

Francesco
--
http://www.erlang-consulting.com

G Bulmer wrote:
> After last weeks excellent Erlang workshop, I am thinking about going  
> to the 13th International Erlang/OTP User Conference.
> Unfortunately, I don't know where it is (more precisely than Stockholm).
>
> Is it in a hotel or a conference centre?
> Is there a hotel offering a good deal on accommodation? I notice that  
> Hotell ?lvsj? was offering a discount last year.
>
> I realise this is very likely too late for this year, but I will  
> suggest it anyway.
> It takes several days out of my schedule to go to a conference, and  
> it's much easier to justify to myself when there are a couple of days  
> of content. I would be very interested in a full day of training- 
> orientated sessions towards "Erlang for those who have largely read  
> and understood Joe's book". The things I feel I miss are the  
> idiomatic uses of Erlang (like John Hughes' tiny [X || X>0]), and I'd  
> love a 1/2+ day chunk of "Programming Distributed Erlang  
> Applications: Pitfalls & Recipes" with hands on exercises. I suspect  
> other newbies would too.
>
> I wouldn't expect it for free, though I am price sensitive (this is  
> my holiday, after all :-), maybe 116 ?/person, with a group of 20  
> people? We bring our laptops with Erlang installed and ready to go,  
> and the organiser provides a wireless network to distribute example  
> code, slides and stuff. The whole thing could be the day after the  
> user conference.
>
> If Erlang takes off big, it will be almost essential for next year.  
> So the opportunity is to beta-test the whole process on us this year :-)
>
> I notice that last year, the conference was 2 days long. Does anyone  
> know why it's only one day, the 8th, this year, or is there more  
> announcements to come?
>
> G Bulmer
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>
>   



From ulf.wiger@REDACTED  Thu Oct 11 18:20:02 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Thu, 11 Oct 2007 18:20:02 +0200
Subject: [erlang-questions] erlang bigger than smalltalk?
Message-ID: <470E4D32.1050608@ericsson.com>


I played with Google trends. I know one should be
careful about drawing conclusions from it, but...

http://www.google.com/trends?q=smalltalk%2C+erlang
http://www.google.com/trends?q=erlang%2C+ocaml

I also noted that Joe's book is ranked #86 on the
amazon.com bestseller list (category: computers & internet)

http://www.amazon.com/gp/bestsellers/books/5/ref=pd_ts_pg_4/002-7970867-4196824?ie=UTF8&pg=4

2007 has been a good year for Erlang. (:

BR,
Ulf W


From chris.newcombe@REDACTED  Thu Oct 11 19:07:17 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 10:07:17 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the Conventional
	Web Wisdom"
Message-ID: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>

Ulf wrote:

   >>2007 has been a good year for Erlang

I thought people might be interested in this...

The bi-annual conference on High Performance Transaction Systems
(sponsored by Amazon, Google, eBay and others), took place this week.

  http://www.hpts.ws

Margo Seltzer gave a talk entitled:

  "Erlang plus BDB: Disrupting the Conventional Web Wisdom"

Margo is an original author of Berkeley DB, founder and CTO of
Sleepycat Software (maintainers of Berkeley DB, acquired by Oracle
around two years ago), and is currently an architect at Oracle.  She
is also Herchel Smith Professor of Computer Science at Harvard
University.

None of the presentations are online yet, but I've pasted the abstract
of Margo's talk below.

FYI you can download EDTK and the BDB driver from:

   http://www.snookles.com/erlang/edtk/

Chris

>From http://www.hpts.ws/hpts_abstracts2007.pdf :

 Tuesday 11:00AM ? 11:30AM Margo Seltzer Oracle/Harvard

  Erlang plus BDB: Disrupting the Conventional Web Wisdom.

  "The conventional wisdom says that you implement a web site with web
servers, application servers, and a relational database backend.
However, this solution is neither scalable nor reliable. Next
generation high performance, highly reliable web sites are going to be
implemented using Erlang and Berkeley DB.

Erlang is a language, developed by Ericsson, used in mission-critical
telecom applications requiring nine 9's -- yes, that's 31 ms/year of
downtime. Erlang applications achieve non-stop operation, live
upgrades, and failure resilience, by taking the historic UNIX model of
small, simple processes to the extreme.
Applications consist of thousands or tens of thousands of isolated,
light-weight processes, communicating through message-passing. The
magic is that Erlang process creation and IPC are fast (a few
microseconds), but there is no shared memory, no mutexes, and all
Erlang data are immutable.

The only thing missing from Erlang is large-scale persistent storage.
Enter Berkeley DB. Berkeley DB is an embedded database library
providing ACID, replication, and automatic failover. The Berkeley DB
Driver in the Erlang Driver Toolkit unites the scalable,
fault-resilient, and highly concurrent Erlang
environment with the high-performance, highly- available, flexible
storage of Berkeley DB. This combination enables the development of
enormous, high-performance web sites with the reliability profile of
mission-critical telecom switches."


From dbt@REDACTED  Thu Oct 11 19:23:14 2007
From: dbt@REDACTED (David Terrell)
Date: Thu, 11 Oct 2007 12:23:14 -0500
Subject: [erlang-questions] Difference between gen_server and gen_fsm
In-Reply-To: <E14B0D03-25E6-43F9-A6A0-4D50508613D0@geophile.com>
References: <E14B0D03-25E6-43F9-A6A0-4D50508613D0@geophile.com>
Message-ID: <20071011172314.GP11915@sphinx.chicagopeoplez.org>

On Wed, Oct 10, 2007 at 10:00:02PM -0400, Jack Orenstein wrote:
> I'm going through the OTP design principles book and think that I am  
> missing the point of gen_fsm. The difference between gen_fsm and  
> gen_server seems to be a small syntactic one. Instead of handle_call  
> handling each kind of request, there is a function per FSM state. Not  
> having used gen_fsm yet, it seems that gen_fsm would lead to cleaner  
> code. But the two behaviors are so similar, I feel that I am either  
> missing the point of gen_fsm, or completely missing some aspect of  
> its behavior. Can someone please explain the distinction?

A gen_server should always respond to messages the same way. (not
necessarily exactly the same response, obviously).

A gen_fsm can respond to the same message in different ways depending
on its state, or not accept it at all.

-- 
David Terrell
dbt@REDACTED
((meatspace)) http://meat.net/


From jao@REDACTED  Thu Oct 11 19:23:32 2007
From: jao@REDACTED (Jack Orenstein)
Date: Thu, 11 Oct 2007 13:23:32 -0400 (EDT)
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
Message-ID: <31908.207.190.221.98.1192123412.squirrel@geophile.com>

On Thu, October 11, 2007 1:07 pm, Chris Newcombe wrote:
> ...
> Margo Seltzer gave a talk entitled:
>
>   "Erlang plus BDB: Disrupting the Conventional Web Wisdom"
>
> ... I've pasted the abstract of Margo's talk below.
> ...
>   Erlang plus BDB: Disrupting the Conventional Web Wisdom.
>
>   "The conventional wisdom says that you implement a web site with web
> servers, application servers, and a relational database backend.
> However, this solution is neither scalable nor reliable. Next
> generation high performance, highly reliable web sites are going to be
> implemented using Erlang and Berkeley DB.

Can anyone comment on how dets and BDB compare on performance, scalability,
reliability, and utilities (e.g. bulk load, import/export, ...)

Jack Orenstein



From gbulmer@REDACTED  Thu Oct 11 19:47:51 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Thu, 11 Oct 2007 18:47:51 +0100
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
In-Reply-To: <470E0474.8020104@erlang-consulting.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
	<470E0474.8020104@erlang-consulting.com>
Message-ID: <953B0805-6845-4760-BE94-43D63FA4B77D@gmail.com>

Francesco

The OTP course looks interesting, but I must admit, I'll have to  
reorganise things, I am currently planning to be in the states 'till  
6th Nov.

Is the OTP course starting 5th Nov. a hands-on course? The course  
approach isn't clear to me.
Would it be a reasonable level if all my Erlang experience has been  
based on Joe's book, and 'hacking around'?

Has anyone been on the course, who is also familiar with the contents  
of Joe's book? Have you got comments about the course that you could  
share?

Garry

> Hi Garry,
>
> I agree, the Erlang workshop was the best one so far. 100  
> participants at the Commercial Users of Functional Programming  
> (Growing at this rate, the workshop will become larger than  
> ICFP ;-) ) and 60 at the Erlang workshop are real records clearly  
> showing the positive trends in the industrial use of FP.
>
> One of the things we have done for the past three EUCs is provide  
> Erlang and OTP courses in conjunction with the Conference. The week  
> before the conference, we give a 5 day Erlang by Example, and on  
> the week of teh conference, a 4 day OTP course, with a one day  
> break on the Thursday for the conference itself. More information  
> on the courses is available at
>
> http://www.erlang-consulting.com/training/schedule.html
>
> See you at EUC!
>
> Francesco
> --
> http://www.erlang-consulting.com
>
> G Bulmer wrote:
>> After last weeks excellent Erlang workshop, I am thinking about  
>> going  to the 13th International Erlang/OTP User Conference.
>> Unfortunately, I don't know where it is (more precisely than  
>> Stockholm).
>>
>> Is it in a hotel or a conference centre?
>> Is there a hotel offering a good deal on accommodation? I notice  
>> that  Hotell ?lvsj? was offering a discount last year.
>>
>> I realise this is very likely too late for this year, but I will   
>> suggest it anyway.
>> It takes several days out of my schedule to go to a conference,  
>> and  it's much easier to justify to myself when there are a couple  
>> of days  of content. I would be very interested in a full day of  
>> training- orientated sessions towards "Erlang for those who have  
>> largely read  and understood Joe's book". The things I feel I miss  
>> are the  idiomatic uses of Erlang (like John Hughes' tiny [X ||  
>> X>0]), and I'd  love a 1/2+ day chunk of "Programming Distributed  
>> Erlang  Applications: Pitfalls & Recipes" with hands on exercises.  
>> I suspect  other newbies would too.
>>
>> I wouldn't expect it for free, though I am price sensitive (this  
>> is  my holiday, after all :-), maybe 116 ?/person, with a group of  
>> 20  people? We bring our laptops with Erlang installed and ready  
>> to go,  and the organiser provides a wireless network to  
>> distribute example  code, slides and stuff. The whole thing could  
>> be the day after the  user conference.
>>
>> If Erlang takes off big, it will be almost essential for next  
>> year.  So the opportunity is to beta-test the whole process on us  
>> this year :-)
>>
>> I notice that last year, the conference was 2 days long. Does  
>> anyone  know why it's only one day, the 8th, this year, or is  
>> there more  announcements to come?
>>
>> G Bulmer
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
>>
>>
>



From tty.erlang@REDACTED  Thu Oct 11 19:50:43 2007
From: tty.erlang@REDACTED (t ty)
Date: Thu, 11 Oct 2007 13:50:43 -0400
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <31908.207.190.221.98.1192123412.squirrel@geophile.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
Message-ID: <290b3ba10710111050h7d23c56bi3466c6abc075851a@mail.gmail.com>

A good start is "Evaluation of Database Management Systems for Erlang"
by Emil Hellman. See
http://www.erlang-consulting.com/aboutus/erlangarticles.html

t

On 10/11/07, Jack Orenstein <jao@REDACTED> wrote:
> Can anyone comment on how dets and BDB compare on performance, scalability,
> reliability, and utilities (e.g. bulk load, import/export, ...)
>


From francesco@REDACTED  Thu Oct 11 20:05:57 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Thu, 11 Oct 2007 18:05:57 -0000
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
 Conference going to be?
In-Reply-To: <953B0805-6845-4760-BE94-43D63FA4B77D@gmail.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
	<470E0474.8020104@erlang-consulting.com>
	<953B0805-6845-4760-BE94-43D63FA4B77D@gmail.com>
Message-ID: <4737528C.4040108@erlang-consulting.com>


>
> The OTP course looks interesting, but I must admit, I'll have to 
> reorganise things, I am currently planning to be in the states 'till 
> 6th Nov.
The next scheduled ones are in London in November and South Africa in 
December.
>
> Is the OTP course starting 5th Nov. a hands-on course? The course 
> approach isn't clear to me.
Very much hands on. About 50% of the time is dedicated to exercises. 
They consists in building a banking system, where the gen_server is the 
database and the FSM is the ATM machine. (Web banking APIs and the ATM 
graphics are provided).
> Would it be a reasonable level if all my Erlang experience has been 
> based on Joe's book, and 'hacking around'?
Yes. Everyone at all levels (Assuming they have a programming background 
and have used Erlang) who attends an OTP course will get out the 
knowledge needed based on their entry level. This will vary from the 
person who just attended an Erlang course the week before to the 
developer who has worked with Erlang for over a year and is now making 
sure he has not missed anything. Only issue with this course is that 5 
days will be compressed into 4, so the tempo will be higher. 
> Has anyone been on the course, who is also familiar with the contents 
> of Joe's book? Have you got comments about the course that you could 
> share?
I need to leave that to others who attended.

Regards,
Francesco
--
http://www.erlang-consulting.com

>
> Garry
>
>> Hi Garry,
>>
>> I agree, the Erlang workshop was the best one so far. 100 
>> participants at the Commercial Users of Functional Programming 
>> (Growing at this rate, the workshop will become larger than ICFP ;-) 
>> ) and 60 at the Erlang workshop are real records clearly showing the 
>> positive trends in the industrial use of FP.
>>
>> One of the things we have done for the past three EUCs is provide 
>> Erlang and OTP courses in conjunction with the Conference. The week 
>> before the conference, we give a 5 day Erlang by Example, and on the 
>> week of teh conference, a 4 day OTP course, with a one day break on 
>> the Thursday for the conference itself. More information on the 
>> courses is available at
>>
>> http://www.erlang-consulting.com/training/schedule.html
>>
>> See you at EUC!
>>
>> Francesco
>> -- 
>> http://www.erlang-consulting.com
>>
>> G Bulmer wrote:
>>> After last weeks excellent Erlang workshop, I am thinking about 
>>> going  to the 13th International Erlang/OTP User Conference.
>>> Unfortunately, I don't know where it is (more precisely than 
>>> Stockholm).
>>>
>>> Is it in a hotel or a conference centre?
>>> Is there a hotel offering a good deal on accommodation? I notice 
>>> that  Hotell ?lvsj? was offering a discount last year.
>>>
>>> I realise this is very likely too late for this year, but I will  
>>> suggest it anyway.
>>> It takes several days out of my schedule to go to a conference, and  
>>> it's much easier to justify to myself when there are a couple of 
>>> days  of content. I would be very interested in a full day of 
>>> training- orientated sessions towards "Erlang for those who have 
>>> largely read  and understood Joe's book". The things I feel I miss 
>>> are the  idiomatic uses of Erlang (like John Hughes' tiny [X || 
>>> X>0]), and I'd  love a 1/2+ day chunk of "Programming Distributed 
>>> Erlang  Applications: Pitfalls & Recipes" with hands on exercises. I 
>>> suspect  other newbies would too.
>>>
>>> I wouldn't expect it for free, though I am price sensitive (this is  
>>> my holiday, after all :-), maybe 116 ?/person, with a group of 20  
>>> people? We bring our laptops with Erlang installed and ready to go,  
>>> and the organiser provides a wireless network to distribute example  
>>> code, slides and stuff. The whole thing could be the day after the  
>>> user conference.
>>>
>>> If Erlang takes off big, it will be almost essential for next year.  
>>> So the opportunity is to beta-test the whole process on us this year 
>>> :-)
>>>
>>> I notice that last year, the conference was 2 days long. Does 
>>> anyone  know why it's only one day, the 8th, this year, or is there 
>>> more  announcements to come?
>>>
>>> G Bulmer
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>>
>>
>
>
>



From bent@REDACTED  Thu Oct 11 20:11:08 2007
From: bent@REDACTED (Ben Munat)
Date: Thu, 11 Oct 2007 08:11:08 -1000
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <31908.207.190.221.98.1192123412.squirrel@geophile.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
Message-ID: <470E673C.2090303@munat.com>

Jack Orenstein wrote:
> On Thu, October 11, 2007 1:07 pm, Chris Newcombe wrote:
>> ...
>> Margo Seltzer gave a talk entitled:
>>   "Erlang plus BDB: Disrupting the Conventional Web Wisdom"
>>
>>   "The conventional wisdom says that you implement a web site with web
>> servers, application servers, and a relational database backend.
>> However, this solution is neither scalable nor reliable. Next
>> generation high performance, highly reliable web sites are going to be
>> implemented using Erlang and Berkeley DB.
> 
> Can anyone comment on how dets and BDB compare on performance, scalability,
> reliability, and utilities (e.g. bulk load, import/export, ...)


For that matter, what about mnesia? It seems pretty self-serving for her to say 
"The only thing missing from Erlang is large-scale persistent storage". Mnesia 
scales well and is used in high-demand production environments, right?

b


From ulf.wiger@REDACTED  Thu Oct 11 20:30:04 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Thu, 11 Oct 2007 20:30:04 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <470E673C.2090303@munat.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com>
Message-ID: <470E6BAC.3020707@ericsson.com>

Ben Munat wrote:
> Jack Orenstein wrote:
>> On Thu, October 11, 2007 1:07 pm, Chris Newcombe wrote:
>>> ...
>>> Margo Seltzer gave a talk entitled:
>>>   "Erlang plus BDB: Disrupting the Conventional Web Wisdom"
>>>
>>>   "The conventional wisdom says that you implement a web site with web
>>> servers, application servers, and a relational database backend.
>>> However, this solution is neither scalable nor reliable. Next
>>> generation high performance, highly reliable web sites are going to be
>>> implemented using Erlang and Berkeley DB.
>> Can anyone comment on how dets and BDB compare on performance, scalability,
>> reliability, and utilities (e.g. bulk load, import/export, ...)
> 
> 
> For that matter, what about mnesia? It seems pretty self-serving for
> her to say "The only thing missing from Erlang is large-scale
> persistent storage". Mnesia scales well and is used in high-demand
> production environments, right?

To be fair, mnesia is not intended for very large databases.
Some gigabytes is ok, but beyond that, it is probably wise
to start looking at alternatives.

BR,
Ulf W


From ulf.wiger@REDACTED  Thu Oct 11 20:34:22 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Thu, 11 Oct 2007 20:34:22 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <31908.207.190.221.98.1192123412.squirrel@geophile.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
Message-ID: <470E6CAE.2070404@ericsson.com>

Jack Orenstein wrote:

> Can anyone comment on how dets and BDB compare on performance,
> scalability,  reliability, and utilities (e.g. bulk load,
 > import/export, ...)

One thing dets has against it is that it is designed to
support both keys and data of any type - even mixed types
in the same table. This is fairly unusual, and a serious
impediment when speed is of the essence. For example,
the dets implementation can never know how large a given
object is before it starts reading it.

I think it would be interesting to consider the option
to add type restrictions to selected mnesia tables. This
could be used to optimize the disk storage, e.g. by
choosing BDB or some other back-end.

BR,
Ulf W


From francesco@REDACTED  Thu Oct 11 20:35:54 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Thu, 11 Oct 2007 18:35:54 -0000
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <470E673C.2090303@munat.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com>
Message-ID: <47375990.4050504@erlang-consulting.com>


> For that matter, what about mnesia? It seems pretty self-serving for her to say 
> "The only thing missing from Erlang is large-scale persistent storage". Mnesia 
> scales well and is used in high-demand production environments, right?
>   
Yes, but Mnesia has its limits. The paper pointed out by Tee 
(http://erlang-consulting.com/thesis/dbms_eval.html) was researched when 
we had to track about 200 million chickens a year (please do not laugh, 
I am serious :-) ) undergoing about 5 or 6 state/ownership transitions. 
That is where postgres, BDB and other hardcore databases come into the 
picture. Mnesia is excellent for caching and storing live data, but not 
for archiving or large quantities of rarely used data.


Regards,
Francesco
--
http://www.erlang-consulting.com


From gbulmer@REDACTED  Thu Oct 11 20:37:12 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Thu, 11 Oct 2007 19:37:12 +0100
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
In-Reply-To: <4737528C.4040108@erlang-consulting.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
	<470E0474.8020104@erlang-consulting.com>
	<953B0805-6845-4760-BE94-43D63FA4B77D@gmail.com>
	<4737528C.4040108@erlang-consulting.com>
Message-ID: <87AEE972-73E9-4EA1-8AB4-E8C7C668AA5C@gmail.com>

Francesco, thank you for your help.

On 11 Nov 2007, at 19:05, Francesco Cesarini wrote:
>
>>
>> The OTP course looks interesting, but I must admit, I'll have to  
>> reorganise things, I am currently planning to be in the states  
>> 'till 6th Nov.
> The next scheduled ones are in London in November and South Africa  
> in December.
Unfortunately, I am currently committed to seeing a very dear friend  
for Loy Krathong during the week of the London course, and South  
Africa is a bit of a stretch for my own interest, even for me!
>>
>> Is the OTP course starting 5th Nov. a hands-on course? The course  
>> approach isn't clear to me.
> Very much hands on. About 50% of the time is dedicated to  
> exercises. They consists in building a banking system, where the  
> gen_server is the database and the FSM is the ATM machine. (Web  
> banking APIs and the ATM graphics are provided).
That sounds very interesting.

>> Would it be a reasonable level if all my Erlang experience has  
>> been based on Joe's book, and 'hacking around'?
> Yes. Everyone at all levels (Assuming they have a programming  
> background and have used Erlang) who attends an OTP course will get  
> out the knowledge needed based on their entry level. This will vary  
> from the person who just attended an Erlang course the week before  
> to the developer who has worked with Erlang for over a year and is  
> now making sure he has not missed anything. Only issue with this  
> course is that 5 days will be compressed into 4, so the tempo will  
> be higher.
Can you help my giving some suggestions on useful preparations to do  
ahead of time? A little bit of brushing up over then next few weeks  
might reduce the impact of that compression. Based on what you've  
said, Joe's Mnesia and gen server chapters would be useful.

>> Has anyone been on the course, who is also familiar with the  
>> contents of Joe's book? Have you got comments about the course  
>> that you could share?
> I need to leave that to others who attended.
Okay, fair comment.

There are chapters on OTP, and Mnesia. How would you rate the breadth  
and depth of the course vs the contents of the book? For example, do  
you think the course fills in many of the details and also practices  
it, and/or do you feel the course is much broader? Or is there some  
other benefit?

>
> Regards,
> Francesco
> --
> http://www.erlang-consulting.com
>
>>
>> Garry
>>
>>> Hi Garry,
>>>
>>> I agree, the Erlang workshop was the best one so far. 100  
>>> participants at the Commercial Users of Functional Programming  
>>> (Growing at this rate, the workshop will become larger than  
>>> ICFP ;-) ) and 60 at the Erlang workshop are real records clearly  
>>> showing the positive trends in the industrial use of FP.
>>>
>>> One of the things we have done for the past three EUCs is provide  
>>> Erlang and OTP courses in conjunction with the Conference. The  
>>> week before the conference, we give a 5 day Erlang by Example,  
>>> and on the week of teh conference, a 4 day OTP course, with a one  
>>> day break on the Thursday for the conference itself. More  
>>> information on the courses is available at
>>>
>>> http://www.erlang-consulting.com/training/schedule.html
>>>
>>> See you at EUC!
>>>
>>> Francesco
>>> -- 
>>> http://www.erlang-consulting.com
>>>
>>> G Bulmer wrote:
>>>> After last weeks excellent Erlang workshop, I am thinking about  
>>>> going  to the 13th International Erlang/OTP User Conference.
>>>> Unfortunately, I don't know where it is (more precisely than  
>>>> Stockholm).
>>>>
>>>> Is it in a hotel or a conference centre?
>>>> Is there a hotel offering a good deal on accommodation? I notice  
>>>> that  Hotell ?lvsj? was offering a discount last year.
>>>>
>>>> I realise this is very likely too late for this year, but I  
>>>> will  suggest it anyway.
>>>> It takes several days out of my schedule to go to a conference,  
>>>> and  it's much easier to justify to myself when there are a  
>>>> couple of days  of content. I would be very interested in a full  
>>>> day of training- orientated sessions towards "Erlang for those  
>>>> who have largely read  and understood Joe's book". The things I  
>>>> feel I miss are the  idiomatic uses of Erlang (like John Hughes'  
>>>> tiny [X || X>0]), and I'd  love a 1/2+ day chunk of "Programming  
>>>> Distributed Erlang  Applications: Pitfalls & Recipes" with hands  
>>>> on exercises. I suspect  other newbies would too.
>>>>
>>>> I wouldn't expect it for free, though I am price sensitive (this  
>>>> is  my holiday, after all :-), maybe 116 ?/person, with a group  
>>>> of 20  people? We bring our laptops with Erlang installed and  
>>>> ready to go,  and the organiser provides a wireless network to  
>>>> distribute example  code, slides and stuff. The whole thing  
>>>> could be the day after the  user conference.
>>>>
>>>> If Erlang takes off big, it will be almost essential for next  
>>>> year.  So the opportunity is to beta-test the whole process on  
>>>> us this year :-)
>>>>
>>>> I notice that last year, the conference was 2 days long. Does  
>>>> anyone  know why it's only one day, the 8th, this year, or is  
>>>> there more  announcements to come?
>>>>
>>>> G Bulmer
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>



From ciprian.craciun@REDACTED  Thu Oct 11 21:03:45 2007
From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun)
Date: Thu, 11 Oct 2007 22:03:45 +0300
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
Message-ID: <8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>

    If we got to this subject... I have used Berkeley DB in the past
and I would also like to use it in Erlang.

    My question is how stable and complete is the library for Erlang?
Can it be used for real life applications? Has some one used it?

    Because I have searched for libraries -- I think I found several
of them -- but none seemed to be finished. Also I could not find a
proper usage documentation or at least an API. (Please excuse me if I
am wrong. I am a lousy searcher... :) )

    Thanks all,
    Ciprian.


From chris.newcombe@REDACTED  Thu Oct 11 21:10:14 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 12:10:14 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <290b3ba10710111050h7d23c56bi3466c6abc075851a@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<290b3ba10710111050h7d23c56bi3466c6abc075851a@mail.gmail.com>
Message-ID: <781dd98c0710111210t83cbeedj695b37cf2f83def7@mail.gmail.com>

On 10/11/07, t ty <tty.erlang@REDACTED> wrote:
> A good start is "Evaluation of Database Management Systems for Erlang"
> by Emil Hellman. See
> http://www.erlang-consulting.com/aboutus/erlangarticles.html

Unfortunately that report is seriously out of date, with respect to
Erlang and BDB anyway.  It talks about EDTK 1.0 which supported a
small part of the BDB v4.0 API (not even including transactions), and
that version of BDB is now more than 5 years old.  It also contains
some inaccuracies about even that version of BDB.

The current EDTK (v1.5.1) supports almost the entire BDB v4.5.20 API,
including transactions, replication.  It also has a number of new
convienience layers (e.g. easier configuration of BDB and helpers for
replication), and has been extensively stress tested and performance
tested in an industrial setting over several months.

As to inaccuracies:

e.g. In the 'safety' section it says:

  "For Berkeley DB, logging of operations is left as a responsibility
for the application. "

... which is simply incorrect (unless it means setting the single flag
that enables that feature, as transaction support is optional).

It then says:

  "Berkeley DB was considered slightly worse than the others because
of its reliance on the applications using it."

This may be fair, because Berkeley DB is an embedded database, i.e.
linked into your application (and when used via the EDTK defaults,
dyamically linked in by the Erlang VM process).  i.e. There is no ODBC
driver in the mix -- BDB is exposed as an Erlang port.  This gives big
performance advantages, but the application does need to obey some
rules, such as calling the 'recovery' API at startup.

Fortunately the new version of EDTK does all that for you.  See:

   http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-cnewcom

Chris


From chris.newcombe@REDACTED  Thu Oct 11 21:29:22 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 12:29:22 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>
Message-ID: <781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>

I'd recommend using EDTK 1.5.1, available from here:

   http://www.snookles.com/erlang/edtk/


>    My question is how stable and complete is the library for Erlang?

Obviously I'm biased as I put a lot of work into updating EDTK
precisely to make BDB support 'industrial strength'.

I've done a lot of testing (on linux only) and it appears to be stable
(indeed it comes with about 20 official patches for Berkeley DB from
Oracle to fix bugs that I found during stress testing).

It is also mostly complete; all major BDB subsystems are supported,
including transactions, replication, full configuration, the 'stats'
APIs (which return internal metadata that is vital for performance
tuning etc), distributed transactions (including a ready-to-roll
'Global Transaction Manager' server), and even replicated distributed
transactions :)

The test-suite is also quite comprehensive.  Sadly some of the tests
contain timers and timeouts that are somewhat hardware dependent -- it
passes on my test systems, but fails on machines that differ
significantly from those machines in performance.

> Can it be used for real life applications? Has some one used it?

There are people building industrial systems with it, but
unfortunately can't give more details.  Also, the Berkeley DB team at
Oracle is investigating whether to use it to enhance their own testing
of BDB-HA (replication), as Erlang makes distributed tests so much
easier than other approaches.

> Also I could not find a proper usage documentation or at least an API.

Here is a fairly detailed introduction:

  http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-cnewcom

There is also significant API documentation at the top of most of the
modules in examples/berkeley_db, all in the tarball.    (Also there
are two stand alone short examples, with and without replication, that
both have documentation.)

  http://www.snookles.com/erlang/edtk/edtk-1.5.1.tar.gz

Here is some rationale and detail for the design and changes to EDTK

  http://www.snookles.com/erlang/edtk/EDTK_BerkeleyDB.pdf

Chris


On 10/11/07, Ciprian Dorin Craciun <ciprian.craciun@REDACTED> wrote:
>    If we got to this subject... I have used Berkeley DB in the past
> and I would also like to use it in Erlang.
>
>    My question is how stable and complete is the library for Erlang?
> Can it be used for real life applications? Has some one used it?
>
>    Because I have searched for libraries -- I think I found several
> of them -- but none seemed to be finished. Also I could not find a
> proper usage documentation or at least an API. (Please excuse me if I
> am wrong. I am a lousy searcher... :) )
>
>    Thanks all,
>    Ciprian.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ciprian.craciun@REDACTED  Thu Oct 11 21:40:02 2007
From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun)
Date: Thu, 11 Oct 2007 22:40:02 +0300
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>
	<781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>
Message-ID: <8e04b5820710111240n18f5102et2ab9c4969ff4aa82@mail.gmail.com>

    Thank you for the prompt and complete reply! I will investigate
the library, as I am eager to try it.

    Ciprian.


On 10/11/07, Chris Newcombe <chris.newcombe@REDACTED> wrote:
> I'd recommend using EDTK 1.5.1, available from here:
>
>    http://www.snookles.com/erlang/edtk/
>
>
> >    My question is how stable and complete is the library for Erlang?
>
> Obviously I'm biased as I put a lot of work into updating EDTK
> precisely to make BDB support 'industrial strength'.
>
> I've done a lot of testing (on linux only) and it appears to be stable
> (indeed it comes with about 20 official patches for Berkeley DB from
> Oracle to fix bugs that I found during stress testing).
>
> It is also mostly complete; all major BDB subsystems are supported,
> including transactions, replication, full configuration, the 'stats'
> APIs (which return internal metadata that is vital for performance
> tuning etc), distributed transactions (including a ready-to-roll
> 'Global Transaction Manager' server), and even replicated distributed
> transactions :)
>
> The test-suite is also quite comprehensive.  Sadly some of the tests
> contain timers and timeouts that are somewhat hardware dependent -- it
> passes on my test systems, but fails on machines that differ
> significantly from those machines in performance.
>
> > Can it be used for real life applications? Has some one used it?
>
> There are people building industrial systems with it, but
> unfortunately can't give more details.  Also, the Berkeley DB team at
> Oracle is investigating whether to use it to enhance their own testing
> of BDB-HA (replication), as Erlang makes distributed tests so much
> easier than other approaches.
>
> > Also I could not find a proper usage documentation or at least an API.
>
> Here is a fairly detailed introduction:
>
>   http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-cnewcom
>
> There is also significant API documentation at the top of most of the
> modules in examples/berkeley_db, all in the tarball.    (Also there
> are two stand alone short examples, with and without replication, that
> both have documentation.)
>
>   http://www.snookles.com/erlang/edtk/edtk-1.5.1.tar.gz
>
> Here is some rationale and detail for the design and changes to EDTK
>
>   http://www.snookles.com/erlang/edtk/EDTK_BerkeleyDB.pdf
>
> Chris
>
>
> On 10/11/07, Ciprian Dorin Craciun <ciprian.craciun@REDACTED> wrote:
> >    If we got to this subject... I have used Berkeley DB in the past
> > and I would also like to use it in Erlang.
> >
> >    My question is how stable and complete is the library for Erlang?
> > Can it be used for real life applications? Has some one used it?
> >
> >    Because I have searched for libraries -- I think I found several
> > of them -- but none seemed to be finished. Also I could not find a
> > proper usage documentation or at least an API. (Please excuse me if I
> > am wrong. I am a lousy searcher... :) )
> >
> >    Thanks all,
> >    Ciprian.


From chris.newcombe@REDACTED  Thu Oct 11 21:44:17 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 12:44:17 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470E673C.2090303@munat.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com>
Message-ID: <781dd98c0710111244l30188d24g3f863279108bf233@mail.gmail.com>

> For that matter, what about mnesia? It seems pretty self-serving for her to say
> "The only thing missing from Erlang is large-scale persistent storage". Mnesia
> scales well and is used in high-demand production environments, right?

Mnesia is clearly a fine piece of software, but it was designed for
certain use-cases, and accordingly has different trade-offs when
compared to Berkeley DB.

e.g. Berkeley DB supports 256 TB data per machine (max record size is
4GB) -- the mnesia limit is currently much lower than that (4 GB per
node in 32-bit Erlang?).   I suspect that Margo was refering to this
point (the 'large scale' comment).

Other notable differences:

Mnesia and BDB have different recovery models.  I've read that if an
mnesia/dets node loses power during a disk-write it can reportedly
take a very long time to repair the file (hours?).  Berkeley DB with
transactions can usually recover very quickly (depends on the
configured frequency of checkpoints and rate/size of new transactions,
just like any conventional database with write-ahead logging).

The concurrency models are different: Berkeley DB has page-level
locking, not row-level locking, but concurrency is on a per-OS thread
basis (EDTK adds private configurable threadpools to avoid blocking
the Erlang scheduler threads when using BDB).  Which model is better
depends on use-cases.

The replication models are different. Berkeley DB offers a
single-master, multiple-replica system with fast failover.  It can
handle partitioned networks, assuming that there is still a connected
majority of nodes.  I've read several reports that mnesia doesn't
handle network partitioning very well at all (to the point where it is
best to disable auto-reconnection of Erlang nodes that are running
mnesia), presumably because it was designed for telecoms switches
where network partitioning is relatively rare.

Chris


From yinso.chen@REDACTED  Thu Oct 11 22:07:21 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 11 Oct 2007 13:07:21 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>
	<781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>
Message-ID: <779bf2730710111307x5a35c92dgd4ae0d039a1e870b@mail.gmail.com>

Hi Chris -

this looks great.  For those of us who are more familiar with RDBMS vs. BDB,
how much more abstraction is required to bring the BDB API up to the level
of RDBMS?  I'm not thinking of a SQL frontend to BDB, but more like having a
easy mechanism (e.g. single function call) to create tables, select records,
insert records, etc.

Thanks,
yc

On 10/11/07, Chris Newcombe <chris.newcombe@REDACTED> wrote:
>
> I'd recommend using EDTK 1.5.1, available from here:
>
>    http://www.snookles.com/erlang/edtk/
>
>
> >    My question is how stable and complete is the library for Erlang?
>
> Obviously I'm biased as I put a lot of work into updating EDTK
> precisely to make BDB support 'industrial strength'.
>
> I've done a lot of testing (on linux only) and it appears to be stable
> (indeed it comes with about 20 official patches for Berkeley DB from
> Oracle to fix bugs that I found during stress testing).
>
> It is also mostly complete; all major BDB subsystems are supported,
> including transactions, replication, full configuration, the 'stats'
> APIs (which return internal metadata that is vital for performance
> tuning etc), distributed transactions (including a ready-to-roll
> 'Global Transaction Manager' server), and even replicated distributed
> transactions :)
>
> The test-suite is also quite comprehensive.  Sadly some of the tests
> contain timers and timeouts that are somewhat hardware dependent -- it
> passes on my test systems, but fails on machines that differ
> significantly from those machines in performance.
>
> > Can it be used for real life applications? Has some one used it?
>
> There are people building industrial systems with it, but
> unfortunately can't give more details.  Also, the Berkeley DB team at
> Oracle is investigating whether to use it to enhance their own testing
> of BDB-HA (replication), as Erlang makes distributed tests so much
> easier than other approaches.
>
> > Also I could not find a proper usage documentation or at least an API.
>
> Here is a fairly detailed introduction:
>
>   http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-cnewcom
>
> There is also significant API documentation at the top of most of the
> modules in examples/berkeley_db, all in the tarball.    (Also there
> are two stand alone short examples, with and without replication, that
> both have documentation.)
>
>   http://www.snookles.com/erlang/edtk/edtk-1.5.1.tar.gz
>
> Here is some rationale and detail for the design and changes to EDTK
>
>   http://www.snookles.com/erlang/edtk/EDTK_BerkeleyDB.pdf
>
> Chris
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071011/3bca8dd0/attachment.htm>

From gbulmer@REDACTED  Thu Oct 11 22:09:19 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Thu, 11 Oct 2007 21:09:19 +0100
Subject: [erlang-questions] erlang-questions Digest, Vol 5, Issue 34
In-Reply-To: <mailman.275.1192125958.2732.erlang-questions@erlang.org>
References: <mailman.275.1192125958.2732.erlang-questions@erlang.org>
Message-ID: <A16D87AA-7300-483B-AC2D-29831878BE1C@gmail.com>


Call me cheap, but *anything* which which doesn't tell me the  
commercial price on the web site makes me nervous.
I did look at BDB pricing a few years ago, and I recall we were  
looking at $30k very quickly.
Even that level of pricing seems to have disappeared from the Oracle/ 
sleepycat web site.

I would like something like InterSystems Cach? database, which seems  
to be close in spirit to Erlang.
I have not used it in a live project, but I have spent time with a  
guy using it in anger in large US hospitals (yes, in the care-side,  
not just administration), and he made some very interesting and  
relevant points.
Cache can have multiple schema's for the 'same' 'table' active  
simultaneously, and
It doesn't require the system to be taken down in order to add or  
modify schema's.

I am very out of date with BDB, but I don't think it has those types  
of capabilities (but, I'm very happy to be corrected).

GB

On 11 Oct 2007, at 10:07:17, Chris Newcombe wrote:
> Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
> 	Conventional	Web Wisdom"
> I thought people might be interested in this...
>
> The bi-annual conference on High Performance Transaction Systems
> (sponsored by Amazon, Google, eBay and others), took place this week.
>   http://www.hpts.ws
>
> Margo Seltzer gave a talk entitled: "Erlang plus BDB: Disrupting  
> the Conventional Web Wisdom"
>
> Margo is an original author of Berkeley DB, founder and CTO of
> Sleepycat Software (maintainers of Berkeley DB, acquired by Oracle
> around two years ago), and is currently an architect at Oracle.  ...

> ... the abstract of Margo's talk below.
>
>> From http://www.hpts.ws/hpts_abstracts2007.pdf :
>
>  Tuesday 11:00AM ? 11:30AM Margo Seltzer Oracle/Harvard
>
>   Erlang plus BDB: Disrupting the Conventional Web Wisdom.
>
>   "The conventional wisdom says that you implement a web site with web
> servers, application servers, and a relational database backend.
> However, this solution is neither scalable nor reliable. Next
> generation high performance, highly reliable web sites are going to be
> implemented using Erlang and Berkeley DB.
>
> ... Enter Berkeley DB. Berkeley DB is an embedded database library
> providing ACID, replication, and automatic failover. The Berkeley DB
> Driver in the Erlang Driver Toolkit unites the scalable,
> fault-resilient, and highly concurrent Erlang
> environment with the high-performance, highly- available, flexible
> storage of Berkeley DB. ...


From ulf@REDACTED  Thu Oct 11 22:25:18 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Thu, 11 Oct 2007 22:25:18 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111244l30188d24g3f863279108bf233@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com>
	<781dd98c0710111244l30188d24g3f863279108bf233@mail.gmail.com>
Message-ID: <8209f740710111325t7af24e25tdaa85da255b577a2@mail.gmail.com>

2007/10/11, Chris Newcombe <chris.newcombe@REDACTED>:
>
>  I've read several reports that mnesia doesn't
> handle network partitioning very well at all (to the point where it is
> best to disable auto-reconnection of Erlang nodes that are running
> mnesia), presumably because it was designed for telecoms switches
> where network partitioning is relatively rare.

I think mnesia does offer some decent support for resolving partitioned
networks. Disabling the auto-connect feature isn't mainly to keep
mnesia from messing up - the application will automatically heal as
well, and perhaps start writing to the database while the state of the
data is still undecided. There are also other players: global will
automatically resynch and start deconflicting names, and its default
method of deconflicting is to pick one candidate at random and slay
it (other methods can be specified per-name). Before alternative methods
became available, I had some particularly memorable moments when
the (globally registered) process in charge of trying to resolve the
situation of partioned network was brutally murdered by global at
the worst possible time. (:  This, of course, had nothing to do with
mnesia.

We have also seen, on rare occasions, where particularly bad
network errors have caused the node supervision heartbeat to
time out between some nodes, but not others, making it
extremely difficult to figure out even what parts of the system
are still functioning.

So disabling the automatic reconnect means that you have a fighting
chance of finding out what happened, before the software tries to
organically heal itself. Erlang's auto-connect semantics and distributed
nature are wonderful, most of the time, but can be a real pain in
certain situations. And there is no solution to partitioned networks
that works well in all situations.

One could imagine a few configurable strategies that mnesia could
employ automatically if partitioned network occurs...

BR,
Ulf W


From dmorton@REDACTED  Thu Oct 11 21:52:00 2007
From: dmorton@REDACTED (Damien Morton)
Date: Thu, 11 Oct 2007 15:52:00 -0400
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <47375990.4050504@erlang-consulting.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>
	<47375990.4050504@erlang-consulting.com>
Message-ID: <470E7EE0.4060703@bitfurnace.com>

Is there any reason why Mnesia couldnt use BDB instead of dets?

On 11/11/2007 2:35 PM, Francesco Cesarini wrote:
>> For that matter, what about mnesia? It seems pretty self-serving for her to say 
>> "The only thing missing from Erlang is large-scale persistent storage". Mnesia 
>> scales well and is used in high-demand production environments, right?
>>   
>>     
> Yes, but Mnesia has its limits. The paper pointed out by Tee 
> (http://erlang-consulting.com/thesis/dbms_eval.html) was researched when 
> we had to track about 200 million chickens a year (please do not laugh, 
> I am serious :-) ) undergoing about 5 or 6 state/ownership transitions. 
> That is where postgres, BDB and other hardcore databases come into the 
> picture. Mnesia is excellent for caching and storing live data, but not 
> for archiving or large quantities of rarely used data.
>
>
> Regards,
> Francesco
> --
> http://www.erlang-consulting.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>
>   



From jcone@REDACTED  Thu Oct 11 23:08:14 2007
From: jcone@REDACTED (James Cone)
Date: Fri, 12 Oct 2007 10:08:14 +1300
Subject: [erlang-questions] : Newbie: Is there a short-circuit for the
 comma operator?
In-Reply-To: <mailman.273.1192087573.2732.erlang-questions@erlang.org>
References: <mailman.273.1192087573.2732.erlang-questions@erlang.org>
Message-ID: <470E90BE.5010007@eservglobal.com>

> Shopping-list?
> 
> English is not my native language, so I assume you mean the number of
> function arguments tends to grow to painful numbers. In that case:
> yes, records can help.
> 

Yes, that's what I mean.


From chris.newcombe@REDACTED  Thu Oct 11 23:11:45 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 14:11:45 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <779bf2730710111307x5a35c92dgd4ae0d039a1e870b@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<8e04b5820710111203w432a19d4pc513e63bda383ce3@mail.gmail.com>
	<781dd98c0710111229r5ef86beciaf50880d390942c7@mail.gmail.com>
	<779bf2730710111307x5a35c92dgd4ae0d039a1e870b@mail.gmail.com>
Message-ID: <781dd98c0710111411o17175e4s7cfe37bc2d5d9084@mail.gmail.com>

> how much more abstraction is required to bring the BDB API up to the level
> of RDBMS?

BDB keys and records are just arbitrary byte sequences, so you can do
whatever you like.

It's trivial (and fast) to use term_to_binary and binary_to_term to
encode arbitrary records.  (I'd recommend not using that for keys
though -- would be bad if Ericsson changed the external term format
between releases.)

Caveat: encoding 'real Erlang records' this way will break if you
change the record definition across software versions and don't
upgrade the data simultaneously.  To avoid that, either add a version
number member to all records plus code to read them from BDB as plain
tuples and auto-upgrade old versions to the proper #record, or, store
data as lists of tagged tuples.

For 'simple' cases, I do the latter, with very short tags to avoid
wasting space, and macros in the code that map to the tag names, so
that the code remains readable:

    [{n, Name}, {a, Address}]

in code would appear as:

    [{?USER_NAME_T, Name}, {?USER_ADDRESS_T, Address}]

For more complex stuff (or external compatibility) use XML, or JSON.
You could store them textually, or after being parsed into Erlang
terms.

If your application has a few hard-wired data-access patterns, you
just code for those directly.  If you want to build little custom
query languages you can do that too.  e.g. I have a friend who wrote a
prolog interpretter in Erlang and stored the fact-base and rules in
BDB, so you could run prolog queries on it.

Obviously indexing is vital to performande of many applications. But
'indexes' are just other tables that happen to map secondary keys to
primary keys.  You can maintain indexes manually as part of your
transactions (see the examples_*.erl files from the tarball).  Or you
could write a little framework (a layer/API on top of the current
driver) that supported declarative indexes and queries.  There's
infinite literature on query optimizers to look at :)

The BDB product line includes an excellent XML database with full
XQuery support and declarative indexing.  I haven't looked closely at
supporting this via EDTK yet, but it might not be too hard.  The main
problem is that BDB XML is written in C++ and EDTK currently only
supports C libraries.  So EDTK would have to be compiled as C++ (but
still linkable to the Erlang VM which is pure C), in order to catch
exceptions thrown by BDB XML calls.  But the current general framework
should all work just fine.


> I'm not thinking of a SQL frontend to BDB, but more like having a
> easy mechanism ( e.g. single function call) to create tables, select
> records, insert records, etc.

In BDB parlance an RDBMS 'table' is called a 'database'.   The current
EDTK driver makes this a one-liner.  In fact, this is how you access
an existing database tool

    DB = ?BDB_PC:get_db_handle(BdbPort, bdb_DB_BTREE, "test.db"),

> select records,

BDB supports lookup by primary key or (with btree databases)
key-prefix, so this is trivial:

    MyData = binary_to_term( ?BDB:db_get_data(BdbPort, DB, Txn, Key, []) ),

(BTW, BDB also has a FIFO 'queue' database type for fixed-length
records.  The current EDTK driver contains a sketch (not yet
production code, but close) of how to use that to build persistent
transactional message queues.)

> insert records,

Again, trivial (unless you mean with declarative indexing, triggers,
typed tables etc)

    ?BDB:db_put(BdbPort, DB, void, Key, term_to_binary(Data), []).

Or you can just store raw binaries of course :)

Chris


From tonyg@REDACTED  Thu Oct 11 22:15:56 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Thu, 11 Oct 2007 21:15:56 +0100
Subject: [erlang-questions] EPIPE and TCP sockets
Message-ID: <470E847C.7090409@lshift.net>

Hi all,

If I understand correctly, the BSD sockets interface to TCP lets you 
know when a TCP socket is closed with outbound data unacknowledged, by 
sending the sender SIGPIPE (or returning EPIPE if SIGPIPE is blocked).

Is there a way of getting gen_tcp to let me know if a send is not fully 
completed at the time of socket close? (My initial experiments lead me 
to believe the answer is "no", and inspecting .../common/inet_drv.c's 
use of EPIPE tends to support that conclusion.)

The goal here is to be able to use TCP's reliable-delivery feature to 
get told if the socket closes while unacknowledged sent data exists.

Regards,
   Tony


From chris.newcombe@REDACTED  Thu Oct 11 23:18:14 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 14:18:14 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470E7EE0.4060703@bitfurnace.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com>
Message-ID: <781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>

On 10/11/07, Damien Morton <dmorton@REDACTED> wrote:
> Is there any reason why Mnesia couldnt use BDB instead of dets?

Folks at Synapse (Tony Rogvall, Luke Gorrie before he set off to roam
the world and code in Forth for the OLPC project) did that.

  http://www.erlang.se/euc/04/Synapse.pdf

IIRC, there are some posts on this mailing-list (or perhaps on
lambda-the-ultimate from Luke Gorrie) about the performance they
achieved.

I believe that they used a minimal BDB driver for this.   EDTK takes a
different approach and makes the BDB APIs available in a convenient
form.

It would be fantastic if someone hooked-up mnesia and/or dets to BDB
via the EDTK driver.  However I have no plans to do it myself at the
moment.

Chris


From klacke@REDACTED  Thu Oct 11 23:25:28 2007
From: klacke@REDACTED (Claes Wikstrom)
Date: Thu, 11 Oct 2007 23:25:28 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <470E7EE0.4060703@bitfurnace.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com>
Message-ID: <470E94C8.6080902@hyber.org>

Damien Morton wrote:
> Is there any reason why Mnesia couldnt use BDB instead of dets?
> 

It could, and that would indeed remedy one of the weak points of
mnesia as of today, i.e. dets.
Dets is a fine module, but it is certainly beginning to show
its age. When it was written, 12 years ago, the 4G space limitation
seemed as a minor issue.

We need to do a dets rewrite which addresses the two issues
of size and repair speed. Once those two issues are fixed, I think
mnesia stands up just fine to the rest of the dbs out there. Until then
Mnesia only rock if we're sure that the amount of data in our largest
table is limited to a few Gig.

BDB is certainly a fine DB, but don't forget it's really expensive
unless you're whipping up a db backend for your local high school
or some such.


/klacke



From chris.newcombe@REDACTED  Thu Oct 11 23:27:25 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Thu, 11 Oct 2007 14:27:25 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com>
	<781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
Message-ID: <781dd98c0710111427v2736b05ci3b3959168388cc7@mail.gmail.com>

(replying to myself)

> Folks at Synapse (Tony Rogvall, Luke Gorrie before he set off to roam
> the world and code in Forth for the OLPC project) did that.

I found Luke's post:

   http://lambda-the-ultimate.org/node/1081#comment-11953

Pasting here in case the link ever breaks.   (I hope that's OK Luke,
wherever you are...)

  "Pushing Mnesia
  Just must brag that we're surely the record holders for punishment
of Mnesia :-). We have pushed it to >60M records using a custom
backend of BerkeleyDB stored on a cheap 12-disk SCSI array. Mnesia
scalability is everything in our telecom system since we store one
record per subscriber and lots of operators have tens of millions of
subs.
With the disk arrays we do blow out to 8U of rack space for a full
cluster but it still looks comical compared with the racks and racks
that make up Java/Oracle systems at these sites :-)

  By Luke Gorrie at Tue, 2005-11-08 04:16 P


From yinso.chen@REDACTED  Fri Oct 12 00:12:54 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 11 Oct 2007 15:12:54 -0700
Subject: [erlang-questions] export imported functions?
Message-ID: <779bf2730710111512s4a251189p589b74b7c25fa0ac@mail.gmail.com>

Hi all -

Is it possible to re-export imported functions to create a composite module?


Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071011/65a659e2/attachment.htm>

From per@REDACTED  Fri Oct 12 00:44:17 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 12 Oct 2007 00:44:17 +0200 (CEST)
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <470E847C.7090409@lshift.net>
Message-ID: <200710112244.l9BMiHCt012626@pluto.hedeland.org>

Tony Garnock-Jones <tonyg@REDACTED> wrote:
>
>If I understand correctly, the BSD sockets interface to TCP lets you 
>know when a TCP socket is closed with outbound data unacknowledged, by 
>sending the sender SIGPIPE (or returning EPIPE if SIGPIPE is blocked).

Hm, I don't think that's correct - AFAIK SIGPIPE/EPIPE happens only if
you try to write more data to a socket when the other end is considered
to be closed (either due to receiving FIN or RST, or due to
retransmission timeout).

Closing the socket with unack'ed data normally doesn't do anything in
particular, the close() returns immediately but the kernel holds on to
the data and tries to (re)transmit it according to the standard TCP
logic, and you won't be told if it fails.

You can change the latter behaviour via the SO_LINGER socket option, to
have the close() block until the data is ack'ed or failure/timeout. To
quote an earlier message here: "This is actually available for gen_tcp
too, via the undocumented 'linger' option - whether it really works I
don't know." And thinking about it, I'm not sure *how* it will/would
work - there would need to be a "non-blocking blocking on close()"
(maybe there is) to avoid having the whole runtime block...

--Per Hedeland


From bruce@REDACTED  Fri Oct 12 01:29:41 2007
From: bruce@REDACTED (Bruce Fitzsimons)
Date: Fri, 12 Oct 2007 12:29:41 +1300
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <470E847C.7090409@lshift.net>
References: <470E847C.7090409@lshift.net>
Message-ID: <470EB1E5.4010100@fitzsimons.org>

Tony Garnock-Jones wrote:
> <snip>
> Is there a way of getting gen_tcp to let me know if a send is not fully 
> completed at the time of socket close? (My initial experiments lead me 
> to believe the answer is "no", and inspecting .../common/inet_drv.c's 
> use of EPIPE tends to support that conclusion.)
>
> The goal here is to be able to use TCP's reliable-delivery feature to 
> get told if the socket closes while unacknowledged sent data exists.
>
>   
I saw the same sort of issue in a rant against the jabber/XMPP protocol; 
it treats TCP as "reliable" in an application to application sense. TCP 
isn't, although 99% of the time it is probably close enough. See 
http://about.psyc.eu/Jabber#Closing_Idle_Connections  I'd still like to 
use the SCTP support in Erlang to demonstrate XMPP over SCTP and see 
what sort of improvements it might give, but time/motivation prevents 
me. I don't think it would be a magical fix, but the auto-heartbeating 
alone would be an improvement. Anyway I've drifted off-topic :-)

There is a way using getsockopt to find out if there is a tcp buffer of 
unsent/un-acked data, but I believe it is *linux-specific* (ah, TCP_INFO 
returns struct tcp_info. See /usr/include/linux/tcp.h). If that isn't a 
problem for your application it would appear (completely untested) then 
you can use inet:getopts to invoke this. I've also seen an ioctl that 
can get this information but again it is non-standard.

Cheers,
Bruce




From xushiweizh@REDACTED  Fri Oct 12 04:32:59 2007
From: xushiweizh@REDACTED (shiwei xu)
Date: Fri, 12 Oct 2007 10:32:59 +0800
Subject: [erlang-questions] export imported functions?
In-Reply-To: <779bf2730710111512s4a251189p589b74b7c25fa0ac@mail.gmail.com>
References: <779bf2730710111512s4a251189p589b74b7c25fa0ac@mail.gmail.com>
Message-ID: <c01ea5030710111932t4023f240pa07027d25e438a34@mail.gmail.com>

It's an interesting question. Anyone know the answer differ from this:

foo(Arg1, ..., ArgN) -> imported_module:foo(Arg1, .., ArgN).

?

On 10/12/07, YC <yinso.chen@REDACTED> wrote:
>
> Hi all -
>
> Is it possible to re-export imported functions to create a composite
> module?
>
> Thanks,
> yc
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/70feb689/attachment.htm>

From tobbe@REDACTED  Fri Oct 12 10:00:30 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Fri, 12 Oct 2007 10:00:30 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470E94C8.6080902@hyber.org>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>
	<470E94C8.6080902@hyber.org>
Message-ID: <fen9iu$3ot$1@sea.gmane.org>

Claes Wikstrom wrote:
> Damien Morton wrote:
>> Is there any reason why Mnesia couldnt use BDB instead of dets?
>>
> 
> It could, and that would indeed remedy one of the weak points of
> mnesia as of today, i.e. dets.
> Dets is a fine module, but it is certainly beginning to show
> its age. When it was written, 12 years ago, the 4G space limitation
> seemed as a minor issue.
> 
> We need to do a dets rewrite which addresses the two issues
> of size and repair speed. Once those two issues are fixed, I think
> mnesia stands up just fine to the rest of the dbs out there. 

<wild idea>
Perhaps some of the companies that have an interest in this
could cough up funding for one person to do this?
</wild idea>

--Tobbe


> Until then
> Mnesia only rock if we're sure that the amount of data in our largest
> table is limited to a few Gig.
> 
> BDB is certainly a fine DB, but don't forget it's really expensive
> unless you're whipping up a db backend for your local high school
> or some such.
> 
> 
> /klacke
> 



From ulf.wiger@REDACTED  Fri Oct 12 10:19:53 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Fri, 12 Oct 2007 10:19:53 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
 the	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>
	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>
	<781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
Message-ID: <470F2E29.8020300@ericsson.com>

Chris Newcombe wrote:
> 
> It would be fantastic if someone hooked-up mnesia and/or dets to BDB
> via the EDTK driver.  However I have no plans to do it myself at the
> moment.

This was one of the things I was hoping for when I introduced
a low-level table plug-in API in the rdbms contrib.

Joel R took my implementation and patched mnesia to use the
Amazon EC3 as a mnesia backend.

(At least he studied my code, and then wrote his own -
http://wagerlabs.com/archives/105.html)

I didn't try my hack with BDB, but rather implemented a few
easier plugin types (a disk_log-style table and a file system
mount point).

One of the disadvantages of my approach was that it allows for
no errors in the plug-in code. Any crash in the low-level
access functions will cause mnesia to dump core. If one is
willing to live with this, the API is quite easy to deal with.

BR,
Ulf W


From joelr1@REDACTED  Fri Oct 12 10:59:33 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Fri, 12 Oct 2007 09:59:33 +0100
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
	the	Conventional Web Wisdom"
In-Reply-To: <470F2E29.8020300@ericsson.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>
	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>
	<781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
	<470F2E29.8020300@ericsson.com>
Message-ID: <7BE3CD07-C827-48A0-8826-D3C145D52C70@gmail.com>


On Oct 12, 2007, at 9:19 AM, Ulf Wiger (TN/EAB) wrote:

> Joel R took my implementation and patched mnesia to use the
> Amazon EC3 as a mnesia backend.
> [...]
> I didn't try my hack with BDB, but rather implemented a few
> easier plugin types (a disk_log-style table and a file system
> mount point).

I have other fun things to do (OCaml/Erlang bridge anyone?)
but would be willing to do the work if there's proper sponsorship.
It wold not be too hard, just somewhat tricky.

Kudos to Uffe for his rdbms code!

	Thanks, Joel

--
http://wagerlabs.com







From ulf.wiger@REDACTED  Fri Oct 12 11:25:40 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Fri, 12 Oct 2007 11:25:40 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
 the	Conventional Web Wisdom"
In-Reply-To: <7BE3CD07-C827-48A0-8826-D3C145D52C70@gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>
	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>
	<781dd98c0710111418r4d2866a1s69ec362284bef183@mail.gmail.com>
	<470F2E29.8020300@ericsson.com>
	<7BE3CD07-C827-48A0-8826-D3C145D52C70@gmail.com>
Message-ID: <470F3D94.2010901@ericsson.com>

Joel Reymont wrote:
> 
> On Oct 12, 2007, at 9:19 AM, Ulf Wiger (TN/EAB) wrote:
> 
>> Joel R took my implementation and patched mnesia to use the
>> Amazon EC3 as a mnesia backend.
>> [...]
>> I didn't try my hack with BDB, but rather implemented a few
>> easier plugin types (a disk_log-style table and a file system
>> mount point).
> 
> I have other fun things to do (OCaml/Erlang bridge anyone?)
> but would be willing to do the work if there's proper sponsorship.
> It wold not be too hard, just somewhat tricky.

Of course with _proper_ sponsorship, we should at least make
an attempt at a proper back-end API? Something that could
become a supported part of Mnesia.

The rdbms code is open to critique, as it is Open Source.
The Synapse code and your code aren't, but there is
certainly a lot of experience to bring on board when
aiming for a really solid solution.

The rdbms solution simply copied an internal API, and
added a few functions to control schema changes and
table loading. This shouldn't be too hard to support, but
it may be too limiting, besides the fact that it doesn't
condone errors in the plugin code (OTOH, the update takes
place after commit, so errors really should be considered
fatal, I think). Perhaps some pre-commit hook is needed?

BR,
Ulf W


From dgud@REDACTED  Fri Oct 12 11:48:09 2007
From: dgud@REDACTED (Dan Gudmundsson)
Date: Fri, 12 Oct 2007 11:48:09 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
 the	Conventional Web Wisdom"
In-Reply-To: <fen9iu$3ot$1@sea.gmane.org>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>	<470E94C8.6080902@hyber.org>
	<fen9iu$3ot$1@sea.gmane.org>
Message-ID: <470F42D9.70802@erix.ericsson.se>

The problem is that it requires an amount of mnesia hacking to get it 
right. I didn't like/include RDBMS approch because mnesia currently have
such a lousy support for doing things like this. Plugins needs a major
rewrite. Synapse cheated heavily, it worked for their purpose but wont 
work in another setup.

/Dan

Torbjorn Tornkvist wrote:
> Claes Wikstrom wrote:
>> Damien Morton wrote:
>>> Is there any reason why Mnesia couldnt use BDB instead of dets?
>>>
>> It could, and that would indeed remedy one of the weak points of
>> mnesia as of today, i.e. dets.
>> Dets is a fine module, but it is certainly beginning to show
>> its age. When it was written, 12 years ago, the 4G space limitation
>> seemed as a minor issue.
>>
>> We need to do a dets rewrite which addresses the two issues
>> of size and repair speed. Once those two issues are fixed, I think
>> mnesia stands up just fine to the rest of the dbs out there. 
> 
> <wild idea>
> Perhaps some of the companies that have an interest in this
> could cough up funding for one person to do this?
> </wild idea>
> 
> --Tobbe
> 
> 
>> Until then
>> Mnesia only rock if we're sure that the amount of data in our largest
>> table is limited to a few Gig.
>>
>> BDB is certainly a fine DB, but don't forget it's really expensive
>> unless you're whipping up a db backend for your local high school
>> or some such.
>>
>>
>> /klacke
>>
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 


From ulf.wiger@REDACTED  Fri Oct 12 11:53:46 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Fri, 12 Oct 2007 11:53:46 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
 the	Conventional Web Wisdom"
In-Reply-To: <470F42D9.70802@erix.ericsson.se>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>	<470E94C8.6080902@hyber.org>	<fen9iu$3ot$1@sea.gmane.org>
	<470F42D9.70802@erix.ericsson.se>
Message-ID: <470F442A.9050109@ericsson.com>


Yes, but the dets rewrite suggested by Klacke surely
wouldn't have to affect Mnesia much? Obviously, the new
dets would have to consider BW compatibility...

With a more scalable dets (which doesn't have such bad
recovery times on large data sets), mnesia would
automatically benefit.

BR,
Ulf W

Dan Gudmundsson wrote:
> The problem is that it requires an amount of mnesia hacking to get it 
> right. I didn't like/include RDBMS approch because mnesia currently have
> such a lousy support for doing things like this. Plugins needs a major
> rewrite. Synapse cheated heavily, it worked for their purpose but wont 
> work in another setup.
> 
> /Dan
> 
> Torbjorn Tornkvist wrote:
>> Claes Wikstrom wrote:
>>> Damien Morton wrote:
>>>> Is there any reason why Mnesia couldnt use BDB instead of dets?
>>>>
>>> It could, and that would indeed remedy one of the weak points of
>>> mnesia as of today, i.e. dets.
>>> Dets is a fine module, but it is certainly beginning to show
>>> its age. When it was written, 12 years ago, the 4G space limitation
>>> seemed as a minor issue.
>>>
>>> We need to do a dets rewrite which addresses the two issues
>>> of size and repair speed. Once those two issues are fixed, I think
>>> mnesia stands up just fine to the rest of the dbs out there. 
>> <wild idea>
>> Perhaps some of the companies that have an interest in this
>> could cough up funding for one person to do this?
>> </wild idea>
>>
>> --Tobbe
>>
>>
>>> Until then
>>> Mnesia only rock if we're sure that the amount of data in our largest
>>> table is limited to a few Gig.
>>>
>>> BDB is certainly a fine DB, but don't forget it's really expensive
>>> unless you're whipping up a db backend for your local high school
>>> or some such.
>>>
>>>
>>> /klacke
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From joelr1@REDACTED  Fri Oct 12 12:00:22 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Fri, 12 Oct 2007 11:00:22 +0100
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
	the	Conventional Web Wisdom"
In-Reply-To: <470F42D9.70802@erix.ericsson.se>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>	<470E94C8.6080902@hyber.org>
	<fen9iu$3ot$1@sea.gmane.org> <470F42D9.70802@erix.ericsson.se>
Message-ID: <DF0F5CDF-A4D6-4357-A101-002FE5B0F432@gmail.com>

Dan,

On Oct 12, 2007, at 10:48 AM, Dan Gudmundsson wrote:

> The problem is that it requires an amount of mnesia hacking to get it
> right.

I'll bite! What needs to be done to address the issues of size and  
repair speed?

Doing the work will be much easier once the task is defined!

	Thanks, Joel

--
http://wagerlabs.com







From opendev@REDACTED  Fri Oct 12 12:00:47 2007
From: opendev@REDACTED (Joern)
Date: Fri, 12 Oct 2007 12:00:47 +0200
Subject: [erlang-questions] gen_tcp custom framing
In-Reply-To: <470B39EA.60408@lundata.se>
References: <9e009ad0710011032off2a454leb8204f9d6cf1046@mail.gmail.com>
	<47064E36.708@lundata.se>
	<9e009ad0710080406k5e0ba800q8e9721dc1f68896a@mail.gmail.com>
	<470B39EA.60408@lundata.se>
Message-ID: <9e009ad0710120300k25b53f1dta0eac28986cc0ae2@mail.gmail.com>

Hi Peter,

On 10/9/07, Peter Lund <erlang@REDACTED> wrote:

> 1> f(), <<Z,D:Z/binary,R/binary>> = <<2,3,4,5,6,7,8,9,0 >>, {Z,D,R}.
> {2,<<3,4>>,<<5,6,7,8,9,0>>}
>
> 2> f(), <<Z,D:Z+1/binary,R/binary>> = <<2,3,4,5,6,7,8,9,0 >>, {Z,D,R}.
> ** 1: syntax error before: '+' **
>
> I thought/hoped that 2 above would work, but when testing it, it did not.

IIRC I tried something along those lines using the bitlevel_binaries
compiler flag but to no avail.

> This forces you to do sequencial binary matching:

This is how I do it atm.

Best regards,


rgs/joern
--


From dgud@REDACTED  Fri Oct 12 12:03:45 2007
From: dgud@REDACTED (Dan Gudmundsson)
Date: Fri, 12 Oct 2007 12:03:45 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting
 the	Conventional Web Wisdom"
In-Reply-To: <DF0F5CDF-A4D6-4357-A101-002FE5B0F432@gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	<470E673C.2090303@munat.com>	<47375990.4050504@erlang-consulting.com>	<470E7EE0.4060703@bitfurnace.com>	<470E94C8.6080902@hyber.org>
	<fen9iu$3ot$1@sea.gmane.org> <470F42D9.70802@erix.ericsson.se>
	<DF0F5CDF-A4D6-4357-A101-002FE5B0F432@gmail.com>
Message-ID: <470F4681.9080002@erix.ericsson.se>

Oops sorry wrong thread in thread :-)
replied to plugin-able backends to mnesia.

/Dan

Joel Reymont wrote:
> Dan,
> 
> On Oct 12, 2007, at 10:48 AM, Dan Gudmundsson wrote:
> 
>> The problem is that it requires an amount of mnesia hacking to get it
>> right.
> 
> I'll bite! What needs to be done to address the issues of size and 
> repair speed?
> 
> Doing the work will be much easier once the task is defined!
> 
>     Thanks, Joel
> 
> -- 
> http://wagerlabs.com
> 
> 
> 
> 
> 


From francesco@REDACTED  Fri Oct 12 13:07:12 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Fri, 12 Oct 2007 12:07:12 +0100
Subject: [erlang-questions] 2007 Obfuscated Erlang Competition in
	conjunction with EUC 07
Message-ID: <470F5560.3080807@erlang-consulting.com>

In conjunction with the 13th International Erlang User Conference, an 
Obfuscated Programming Competition will be held. The goal is to write 
the most obfuscated Erlang program, providing a safe forum for poor 
coding practices and programming styles. Through this competition, we 
hope to illustrate some of the subtleties of Erlang and how they can 
best be used and abused. More information is available at:

http://www.erlang-consulting.com/obfuscatederlang.html

The winners will be announced at the Erlang User Conference. The Jury 
(Joe Armstrong, Richard Carlsson, Jan Henry Nystrom) are looking forward 
(read dreading to receive) to your submissions. Be generous!

Francesco
---
http://www.erlang-consulting.com


From grimlog@REDACTED  Fri Oct 12 13:09:12 2007
From: grimlog@REDACTED (Michael Schreckenbauer)
Date: Fri, 12 Oct 2007 13:09:12 +0200
Subject: [erlang-questions] Where is 13th International Erlang/OTP User
	Conference going to be?
In-Reply-To: <4737528C.4040108@erlang-consulting.com>
References: <6BB65B5E-911C-4E35-BAE8-916A98957E81@gmail.com>
	<953B0805-6845-4760-BE94-43D63FA4B77D@gmail.com>
	<4737528C.4040108@erlang-consulting.com>
Message-ID: <200710121309.18046.grimlog@gmx.de>

Hello list, hi Francesco, greetings G Bulmer,

as a new member, I first like to introduce myself.
I'm Michael, my homebase is in the south of germany, in between Salzburg, 
Austria and Munich, Bavaria. My first contact to erlang was about a year ago.
I am married, we have no children, but 5 cats ;)

Am Sonntag, 11. November 2007 schrieb Francesco Cesarini:
> > Has anyone been on the course, who is also familiar with the contents
> > of Joe's book? Have you got comments about the course that you could
> > share?
>
> I need to leave that to others who attended.

I would be one of them. I read Joe's book and attended the course in London in 
June. The course was one of the best I ever was part of. Although my erlang 
knowledge was only basic, based on a beta of the book and some practice I got 
from first small projects in the team I work for, I had no problems 
following. My feeling was, that although the basic idea is to develop a 
simple bank application, the course itself is customized in some ways, 
depending on the needs of people joining and their skills. It's focussed on 
OTP, but we also had a look at traceing and learned a lot about mnesia.
So, I'd say, having worked through the book is enough to get through this 
course and the course is really worth it, if you want to dig deeper into OTP 
and erlang. 
I really enjoyed it. If you have further questions, keep asking :)

> Regards,
> Francesco
> --
> http://www.erlang-consulting.com

Hth and Regards,
Michael


From klacke@REDACTED  Fri Oct 12 14:01:38 2007
From: klacke@REDACTED (Claes Wikstrom)
Date: Fri, 12 Oct 2007 14:01:38 +0200
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
 Conventional Web Wisdom"
In-Reply-To: <c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>	
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>	
	<470E673C.2090303@munat.com>
	<47375990.4050504@erlang-consulting.com>	
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
Message-ID: <470F6222.6050306@hyber.org>

Nigel Head wrote:
> On 10/11/07, *Claes Wikstrom* <klacke@REDACTED 

> 
>     BDB is certainly a fine DB, but don't forget it's really expensive
>     unless you're whipping up a db backend for your local high school
>     or some such.
> 

> 
> Does anyone know how this could / should / has been interpreted in the 
> case of that other slightly less prominent scripting language called 
> erlang ??
> 
> 

Well that is hardly the issue here, the issue is weather I can use
BDB for commercial use. Sure I can if I pay - a lot.

And the "a lot" is also so much so they don't even dare quote the
number on the website. There is nothing wrong with commercial software,
hell I do that for a living. It's just that the problem we as Erlangers have
here isn't that we lack an open source DB, we have one which is just fine.
The problem we have, is that Mnesia ain't that good at storing large
amounts of data - due to dets.

An alternative for Mnesia to BDB, could possibly be gdbm as a storage
backend. We lack a good storage, not a good transaction manager. IMHO.


/klacke





From ingela@REDACTED  Fri Oct 12 14:26:05 2007
From: ingela@REDACTED (Ingela Anderton Andin)
Date: Fri, 12 Oct 2007 14:26:05 +0200
Subject: [erlang-questions] ODBC parameters: unsupported type
In-Reply-To: <mailman.1.1191837602.26409.erlang-questions@erlang.org>
References: <mailman.1.1191837602.26409.erlang-questions@erlang.org>
Message-ID: <470F67DD.4060802@erix.ericsson.se>

Hi!
> Based on http://www.erlang.org/doc/apps/odbc/databases.html#3.3, types such
> as datetime and blob are currently unsupported.  Are there plans to support
> these in the future?  How do people currently enter data with these types,
> besides manually concatenate sql string?
>   

There are currently no plans to add support for additional types.  The 
reason is mainly
no funding  which means it gets low priority. We will probably get 
around to it some time but
if you are really interested in seeing it supported the most effective 
way would be a patch-suggestion with
test cases sent to the erlang-patches mailing list.

Regards Ingela - OTP team


From gbulmer@REDACTED  Fri Oct 12 16:50:03 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Fri, 12 Oct 2007 15:50:03 +0100
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
Message-ID: <2722EBD9-0D75-4655-881D-97FD86DCF7E0@gmail.com>

Newbie question.

> Date: Fri, 12 Oct 2007 12:29:41 +1300
> From: Bruce Fitzsimons <bruce@REDACTED>
> Subject: Re: [erlang-questions] EPIPE and TCP sockets

> ... I'd still like to
> use the SCTP support in Erlang to demonstrate XMPP over SCTP and see
> what sort of improvements it might give, ...

Is the "Erlang SCTP support" likely to give higher message  
throughput, or is this really about reliability guarantees? Is this  
OS sensitive, i.e. very much bound to the quality of SCTP  
implementation for the underlying OS?

G Bulmer



From chris.newcombe@REDACTED  Fri Oct 12 17:31:32 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Fri, 12 Oct 2007 08:31:32 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470E94C8.6080902@hyber.org>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
Message-ID: <781dd98c0710120831y7a85a6e1rc436fc6d7b49ca78@mail.gmail.com>

> BDB is certainly a fine DB, but don't forget it's really expensive
> unless you're whipping up a db backend for your local high school
> or some such.

You can use it for free if you:

  1. open-source your application

or

  2. you write a closed-source application but don't re-distribute it
      (i.e. this allows most classes of internal 'enterprise' use.)

      You must use the definition of 're-distribute' given on this page:

      http://www.oracle.com/technology/software/products/berkeley-db/htdocs/licensing.html

But yes, if you want to use it in closed-source application that you
don't re-distribute, then you probably have to buy a commercial
licence.

I've participated in negotiations for commercial licences for BDB at
two different companies, and in my experience/opinion they were very
flexible and willing to work to find a good deal.  (The final deals
were very, very different.)  That was before they were acquired by
Oracle, but I have no reason to believe that things have changed.

So if you have a use-case that might fall under the commercial
license, then I'd definitely recommend at least asking them about it.
I'd recommend sending any questions to:

	arlene.capsimalis@REDACTED

For the record, I don't work for Oracle, hold any Oracle stock, or
have any financial involvement with them.  I'm just a long-time user
of the product, and happen to think that Erlang and BDB make a great
combination.

Finally, there's absolutely nothing preventing you from downloading
BDB and EDTK and experimenting or prototyping with it...

Chris


From anders.nygren@REDACTED  Fri Oct 12 18:53:50 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 12 Oct 2007 11:53:50 -0500
Subject: [erlang-questions] Building edtk and bdb
Message-ID: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>

Hi
Inspired by the bdb discussion and a current need for a high
performance db, I tried to build the dbd driver. Thanks to Chris
excellent description the build seems to have been successful,
but the regression test fails.

I am not sure if the initial errors are real errors or not. Since they
are testing things that are supposed to fail.

But at the end it definitely fails. I have enclose the test results.
Any suggestions on what could be wrong?

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: regression.out
Type: application/octet-stream
Size: 13411 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/440c3c7f/attachment.obj>

From anders.nygren@REDACTED  Fri Oct 12 19:30:03 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 12 Oct 2007 12:30:03 -0500
Subject: [erlang-questions] Building edtk and bdb
In-Reply-To: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>
References: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>
Message-ID: <d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>

On 10/12/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> Hi
> Inspired by the bdb discussion and a current need for a high
> performance db, I tried to build the dbd driver. Thanks to Chris
> excellent description the build seems to have been successful,
> but the regression test fails.
>
> I am not sure if the initial errors are real errors or not. Since they
> are testing things that are supposed to fail.
>
> But at the end it definitely fails. I have enclose the test results.
> Any suggestions on what could be wrong?
>

Sorry about that, it was an error with some library paths.

But when running the tests now I get timeout errors at different places.
Is that something expected?
I am running on a centrino duo laptop with linux.

/Anders


From chris.newcombe@REDACTED  Fri Oct 12 20:00:45 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Fri, 12 Oct 2007 11:00:45 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470F6222.6050306@hyber.org>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
	<470F6222.6050306@hyber.org>
Message-ID: <781dd98c0710121100w3b469a36mcb0fc75b18117004@mail.gmail.com>

> BDB for commercial use. Sure I can if I pay - a lot.f
>
> And the "a lot" is also so much so they don't even dare quote the
> number on the website.

This is over the top.   The phrase "a lot" is so subjective it is
meaningless.  And in my experience at least, the 'price' isn't on the
website largely because it varies significantly from customer to
customer, in a good way.

e.g. I know from personal experience that Sleepycat were quite happy
to do low up-front costs plus a per-product unit royality, if that's
what suited the customer.  The royality percentage was emminently
reasonable, and depended on the amount of use the product made of BDB
-- i.e. the deal was flexible and fair to both sides.

You can also buy a one-off license to distribute as many copies as you
like, forever.  Obviously that cost considerably more -- but then how
many other *embeddable* (zero-install, zero-human-maintenance),
industrial-strength, performant, highly concurrent, transactional,
replication-ready storage engines are there out there?  You are
getting a very sophisticated piece of equipment for the money.

My knowledge of the the royaltiy deal is from before Oracle bought
Sleepycat, but for all I know they are still this flexible.  And it
only takes one email to them to find out for yourself.  (See my
earlier post today for the email address.)

Chris


From chris.newcombe@REDACTED  Fri Oct 12 20:10:46 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Fri, 12 Oct 2007 11:10:46 -0700
Subject: [erlang-questions] Building edtk and bdb
In-Reply-To: <d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>
References: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>
	<d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>
Message-ID: <781dd98c0710121110t4fa090eal239376a0840ee056@mail.gmail.com>

> But when running the tests now I get timeout errors at different places.
> Is that something expected?
> I am running on a centrino duo laptop with linux.

Alas, yes.   I mentioned this in a post yesterday:

  "The test-suite is also quite comprehensive.  Sadly some of the tests
contain timers and timeouts that are somewhat hardware dependent -- it
passes on my test systems, but fails on machines that differ
significantly from those machines in performance."

If you send me the regression.out file I'll take a look, but I can't
promise anything as I probably won't be able to reproduce your
timeouts on any machines that I have access too (the tests pass on my
machines).

A better solution would be for you to tweak the timeouts yourself, or
add new calls to timer:sleep if necessary.  Please post any changes
that work for you -- if they don't break my tests I would be happy to
include them.

Chris


From anders.nygren@REDACTED  Fri Oct 12 20:19:07 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 12 Oct 2007 13:19:07 -0500
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <781dd98c0710121100w3b469a36mcb0fc75b18117004@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
	<470F6222.6050306@hyber.org>
	<781dd98c0710121100w3b469a36mcb0fc75b18117004@mail.gmail.com>
Message-ID: <d3520a2f0710121119t693cf73cide4997346b3e187@mail.gmail.com>

On 10/12/07, Chris Newcombe <chris.newcombe@REDACTED> wrote:
> > BDB for commercial use. Sure I can if I pay - a lot.f
> >
> > And the "a lot" is also so much so they don't even dare quote the
> > number on the website.
>
> This is over the top.   The phrase "a lot" is so subjective it is
> meaningless.  And in my experience at least, the 'price' isn't on the
> website largely because it varies significantly from customer to
> customer, in a good way.

Well it is on the web site, as a quick google found
http://www.oracle.com/corporate/pricing/technology-price-list.pdf

/Anders


From chris.newcombe@REDACTED  Fri Oct 12 20:36:44 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Fri, 12 Oct 2007 11:36:44 -0700
Subject: [erlang-questions] Building edtk and bdb
In-Reply-To: <781dd98c0710121110t4fa090eal239376a0840ee056@mail.gmail.com>
References: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>
	<d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>
	<781dd98c0710121110t4fa090eal239376a0840ee056@mail.gmail.com>
Message-ID: <781dd98c0710121136o3e387057n954e4a9776a86f9f@mail.gmail.com>

FYI, I know it sucks that the current EDTK 1.5.1 package has some
hard-wired paths in .  I apologise for this -- I just didn't have time
to polish that part, and

Matt Stancliff sent me an 'installer' script a while back which does
the download and applies some patches to remove the absolute paths.
I haven't actually used it myself yet (never had the need), but I have
attached it incase anyone finds it useful.   I will very likely
include it (or the changes it makes) in the next patch release of
EDTK.

I hope this helps,

Chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: edtk-installer-1.5.1.tar.bz2
Type: application/x-bzip2
Size: 3560 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/abd43d62/attachment.bin>

From graemebe@REDACTED  Fri Oct 12 21:13:48 2007
From: graemebe@REDACTED (graemebe)
Date: Fri, 12 Oct 2007 12:13:48 -0700 (PDT)
Subject: [erlang-questions] Learning erlang - is this an efficient way to
 build a list?
Message-ID: <12958075.post@talk.nabble.com>


I've just started learning erlang. I wanted to create a set of processes and
keep their pids in a list and send a message to each with a list
comprehension. In general, is the spawnem function I hacked out below an
efficient way to build a list?

-module(ring).
-export([start/0, recv/0, spawnem/1]).

% Receive a message and print it, repeat.
%
recv() -> 
	receive
		Msg -> io:fwrite("~p\n", [Msg]),
		recv()
	end.

% Spawn some processes and send a message to each.
%
start() ->
	M=10,
	P=ring:spawnem(M),
	[X ! "hello world!" || X <- P].

% Function to Spawn N processes
%
spawnem(N) when is_integer(N), N > 0 -> spawnem_acc(N, [spawn(fun
ring:recv/0)]).
spawnem_acc(N, L) when is_integer(N), N > 1 -> spawnem_acc(N-1, [spawn(fun
ring:recv/0)] ++ L);
spawnem_acc(1, L) -> L.


Any comments/optimisations appreciated!

Cheers,
Graeme.





-- 
View this message in context: http://www.nabble.com/Learning-erlang---is-this-an-efficient-way-to-build-a-list--tf4540288.html#a12958075
Sent from the Erlang Questions mailing list archive at Nabble.com.



From dmercer@REDACTED  Fri Oct 12 21:42:17 2007
From: dmercer@REDACTED (David Mercer)
Date: Fri, 12 Oct 2007 14:42:17 -0500
Subject: [erlang-questions] Learning erlang - is this an efficient way
	to build a list?
In-Reply-To: <12958075.post@talk.nabble.com>
References: <12958075.post@talk.nabble.com>
Message-ID: <008301c80d07$ffac8e50$891ea8c0@SSI.CORP>

You are building the list in reverse order, which is efficient.  I would
typically prepend element X to list L with the syntax [X|L] rather than
[X]++L, but I'm not sure which is more efficient.

Cheers,

David

-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of graemebe
Sent: Friday, October 12, 2007 14:14
To: erlang-questions@REDACTED
Subject: [erlang-questions] Learning erlang - is this an efficient way to
build a list?


I've just started learning erlang. I wanted to create a set of processes and
keep their pids in a list and send a message to each with a list
comprehension. In general, is the spawnem function I hacked out below an
efficient way to build a list?

-module(ring).
-export([start/0, recv/0, spawnem/1]).

% Receive a message and print it, repeat.
%
recv() -> 
	receive
		Msg -> io:fwrite("~p\n", [Msg]),
		recv()
	end.

% Spawn some processes and send a message to each.
%
start() ->
	M=10,
	P=ring:spawnem(M),
	[X ! "hello world!" || X <- P].

% Function to Spawn N processes
%
spawnem(N) when is_integer(N), N > 0 -> spawnem_acc(N, [spawn(fun
ring:recv/0)]).
spawnem_acc(N, L) when is_integer(N), N > 1 -> spawnem_acc(N-1, [spawn(fun
ring:recv/0)] ++ L);
spawnem_acc(1, L) -> L.


Any comments/optimisations appreciated!

Cheers,
Graeme.





-- 
View this message in context:
http://www.nabble.com/Learning-erlang---is-this-an-efficient-way-to-build-a-
list--tf4540288.html#a12958075
Sent from the Erlang Questions mailing list archive at Nabble.com.

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions



From dloutrein.lists@REDACTED  Fri Oct 12 21:43:42 2007
From: dloutrein.lists@REDACTED (denis)
Date: Fri, 12 Oct 2007 15:43:42 -0400
Subject: [erlang-questions] Parameterized modules
Message-ID: <00b201c80d08$32a32030$b029030a@mtl.ubisoft.org>

Hi all,

 

I'm interested in using parameterized modules, but as I didn't find other
documentation except Richard Calsson's paper, I suppose this feature is not
supported.

Is it safe to use it ? Will it be officially supported in future versions ?

 

Many thanks

Denis

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/3d7dfe83/attachment.htm>

From sysop@REDACTED  Fri Oct 12 22:03:41 2007
From: sysop@REDACTED (Matt Stancliff)
Date: Fri, 12 Oct 2007 16:03:41 -0400
Subject: [erlang-questions] Building edtk and bdb
In-Reply-To: <781dd98c0710121136o3e387057n954e4a9776a86f9f@mail.gmail.com>
References: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>
	<d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>
	<781dd98c0710121110t4fa090eal239376a0840ee056@mail.gmail.com>
	<781dd98c0710121136o3e387057n954e4a9776a86f9f@mail.gmail.com>
Message-ID: <31DB65C7-58BF-4FE8-BD87-B65E0EF1E13D@mindspring.com>


On Oct 12, 2007, at 14:36, Chris Newcombe wrote:

> Matt Stancliff sent me an 'installer' script a while back which does
> the download and applies some patches to remove the absolute paths.

   There's a newer version of the edtk+bdb installer at http:// 
mattsmind.net/erlang/

   The new version has nicer options and you can easily compile the bdb
driver natively if you like.  (If you haven't looked, edtk has five  
dependencies
and my install script will download everything and built it for you.)

   Note -- edtk+bdb does not work under OS X.  To get it working with
OS X, gsl and sfl need to be modified (quick #define fixes) and then
something else busts which I never got around to fixing.
   If you get edtk+bdb building under OS X, I would love to see your
patches.

   With all the talk of BDB recently on the mailing list, I'd like to
share some benchmarks I have.  I have a generic BDB/Mnesia compatibility
layer, but it's fairly slow compared to raw BDB or mnesia access:

Tests for entire record DB reading/writing:
Test                    5.5.5 BEAM        Per second
                         (slowdown)
mnesia_dirty_read          1.00            123055.2  (123k)
mnesia_dirty_read_idx      4.04            30451.4   (30k)
mnesia_read_idx            4.16            29570.7   (29k)
mnesia_read                4.27            28836.8   (28k)
mnesia_dirty_write         4.91            25060.5   (25k)
mnesia_write               16.56           7431.8    (7k)
mnesia_dirty_wrte_with_idx 17.03           7224.9    (7k)
mnesia_write_with_idx      27.93           4405.6    (4k)
bdb_record_read            91.3            1347.8    (1k)
bdb_record_read_idx        124.36          989.5     (0.9k)
bdb_record_write           156.15          788       (0.7k)
bdb_record_write_idx       340.41          361.5     (0.3k)


Tests for caching methods:
Test                    5.5.5 BEAM        Per second
                         (slowdown)
ets_read                   1.00            613379.3  (613k)
mnesia_mem_read            1.61            380888.9  (380k)
process_per_cache_item     9.09            67509.4   (67k)
bdb_plain_read             25.09           24449.5   (24k)
bdb_record_read            452.70          1354.9    (1k)


My bdb_record functions act like mnesia functions.  You can do:
   - bdb:write_record(#person{name = bob, age = 23})
   - bdb:read(person, bob) => [#person{name = bob, age = 23}]
Those functions involve a lot of term_to_binary and binary_to_term
conversions.

The bdb_plain_read test is raw BDB reading with a known binary
key (no term conversions).

Note:  Even though my BDB record reading is 450x slower
than reading from ets, I still get over 1300 read/s.


Sincerely,

-Matt
-- 
Matt Stancliff            sysop@REDACTED
AIM: seijimr              iPhone: 678-591-9337
"The best way to predict the future is to invent it." --Alan Kay




From yinso.chen@REDACTED  Fri Oct 12 22:57:01 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 12 Oct 2007 13:57:01 -0700
Subject: [erlang-questions] Learning erlang - is this an efficient way
	to build a list?
In-Reply-To: <12958075.post@talk.nabble.com>
References: <12958075.post@talk.nabble.com>
Message-ID: <779bf2730710121357s45e5d08ewd96d4d0a49397b59@mail.gmail.com>

In general what I've seen is to use the cons operator rather than append,
because append is O(n) and cons is O(1), but in your case it's fine as you
are still reversing the list, and hence your append is O(1) as well.  I
guess the key is in reversing the list ;)

On 10/12/07, graemebe <graemebe@REDACTED> wrote:
>
>
> I've just started learning erlang. I wanted to create a set of processes
> and
> keep their pids in a list and send a message to each with a list
> comprehension. In general, is the spawnem function I hacked out below an
> efficient way to build a list?
>
> -module(ring).
> -export([start/0, recv/0, spawnem/1]).
>
> % Receive a message and print it, repeat.
> %
> recv() ->
>         receive
>                 Msg -> io:fwrite("~p\n", [Msg]),
>                 recv()
>         end.
>
> % Spawn some processes and send a message to each.
> %
> start() ->
>         M=10,
>         P=ring:spawnem(M),
>         [X ! "hello world!" || X <- P].
>
> % Function to Spawn N processes
> %
> spawnem(N) when is_integer(N), N > 0 -> spawnem_acc(N, [spawn(fun
> ring:recv/0)]).
> spawnem_acc(N, L) when is_integer(N), N > 1 -> spawnem_acc(N-1, [spawn(fun
> ring:recv/0)] ++ L);
> spawnem_acc(1, L) -> L.
>
>
> Any comments/optimisations appreciated!
>
> Cheers,
> Graeme.
>
>
>
>
>
> --
> View this message in context:
> http://www.nabble.com/Learning-erlang---is-this-an-efficient-way-to-build-a-list--tf4540288.html#a12958075
> Sent from the Erlang Questions mailing list archive at Nabble.com.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/cc4b31a2/attachment.htm>

From yinso.chen@REDACTED  Fri Oct 12 23:15:46 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 12 Oct 2007 14:15:46 -0700
Subject: [erlang-questions] ODBC parameters: unsupported type
In-Reply-To: <470F67DD.4060802@erix.ericsson.se>
References: <mailman.1.1191837602.26409.erlang-questions@erlang.org>
	<470F67DD.4060802@erix.ericsson.se>
Message-ID: <779bf2730710121415t798a92aex66d5fdfe92e172cc@mail.gmail.com>

On 10/12/07, Ingela Anderton Andin <ingela@REDACTED> wrote:
>
>
> > There are currently no plans to add support for additional types.  The
> > reason is mainly
> > no funding  which means it gets low priority. We will probably get
> > around to it some time but
> > if you are really interested in seeing it supported the most effective
> > way would be a patch-suggestion with
> > test cases sent to the erlang-patches mailing list.


Thanks for the info.  I am still learning about how erlang works in general,
and as I comfortably dig into the internals I will definitely take a look at
the driver.

yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/283eb068/attachment.htm>

From yarivsadan@REDACTED  Sat Oct 13 00:03:20 2007
From: yarivsadan@REDACTED (Yariv Sadan)
Date: Fri, 12 Oct 2007 15:03:20 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <470F6222.6050306@hyber.org>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
	<470F6222.6050306@hyber.org>
Message-ID: <17244f480710121503p4432929alb4c8f5260ee42527@mail.gmail.com>

On 10/12/07, Claes Wikstrom <klacke@REDACTED> wrote:
> Nigel Head wrote:
> > On 10/11/07, *Claes Wikstrom* <klacke@REDACTED
>
> >
> >     BDB is certainly a fine DB, but don't forget it's really expensive
> >     unless you're whipping up a db backend for your local high school
> >     or some such.
> >
>
> >
> > Does anyone know how this could / should / has been interpreted in the
> > case of that other slightly less prominent scripting language called
> > erlang ??
> >
> >
>
> Well that is hardly the issue here, the issue is weather I can use
> BDB for commercial use. Sure I can if I pay - a lot.
>
> And the "a lot" is also so much so they don't even dare quote the
> number on the website. There is nothing wrong with commercial software,
> hell I do that for a living. It's just that the problem we as Erlangers have
> here isn't that we lack an open source DB, we have one which is just fine.
> The problem we have, is that Mnesia ain't that good at storing large
> amounts of data - due to dets.
>
> An alternative for Mnesia to BDB, could possibly be gdbm as a storage
> backend. We lack a good storage, not a good transaction manager. IMHO.
>
>
> /klacke
>

We can use both Postgres and MySQL from Erlang. With ErlyDB, using
those DBMS's is just as Erlangy as Mnesia for most CRUD and SELECT
operations -- you don't have to write SQL by hand. Many big sites run
on MySQL (YouTube, Flickr, Facebook...), so it definitely scales.

In Vimagi, I use MySQL for persistent data and Mnesia for live session
data. It works great, even on a RAM-constrained VPS.

I don't think Erlangers lack options when it comes to high-volume
storage. It's not BDB, Mnesia, or bust :)

Regards,
Yariv


From yinso.chen@REDACTED  Sat Oct 13 00:55:16 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 12 Oct 2007 15:55:16 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <17244f480710121503p4432929alb4c8f5260ee42527@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
	<470F6222.6050306@hyber.org>
	<17244f480710121503p4432929alb4c8f5260ee42527@mail.gmail.com>
Message-ID: <779bf2730710121555l46e944b2j8754301b9aa2a480@mail.gmail.com>

On 10/12/07, Yariv Sadan <yarivsadan@REDACTED> wrote:
>
> > We can use both Postgres and MySQL from Erlang. With ErlyDB, using
> > those DBMS's is just as Erlangy as Mnesia for most CRUD and SELECT
> > operations -- you don't have to write SQL by hand. Many big sites run
> > on MySQL (YouTube, Flickr, Facebook...), so it definitely scales.


Hi Yariv -

A few questions for ya :)

Is the postgresql driver on par with the mysql driver at this time?  Reading
from your blog it seems Postgresql is not yet there.

Would ErlyDB be the DBI/DBD equivalent of the Erlang world?  The nice thing
about open source is that everyone can do what they want, but having a de
facto package for something like this is probably essential.

> In Vimagi, I use MySQL for persistent data and Mnesia for live session
> > data. It works great, even on a RAM-constrained VPS.


Is the live session data a cached data or actually the session table? I
think Mnesia would be great as a cache server (e.g. memcached), but haven't
thought through it, and if you have done so, would love to hear your take.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/98a46df9/attachment.htm>

From ulf.wiger@REDACTED  Sat Oct 13 00:55:53 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Sat, 13 Oct 2007 00:55:53 +0200
Subject: [erlang-questions] Building edtk and bdb
In-Reply-To: <31DB65C7-58BF-4FE8-BD87-B65E0EF1E13D@mindspring.com>
References: <d3520a2f0710120953n489f2c19y2562980cf1fceca3@mail.gmail.com>	<d3520a2f0710121030w469c24dct2c39212a5a29a4c8@mail.gmail.com>	<781dd98c0710121110t4fa090eal239376a0840ee056@mail.gmail.com>	<781dd98c0710121136o3e387057n954e4a9776a86f9f@mail.gmail.com>
	<31DB65C7-58BF-4FE8-BD87-B65E0EF1E13D@mindspring.com>
Message-ID: <470FFB79.9050100@ericsson.com>

Matt Stancliff wrote:
> 
>    With all the talk of BDB recently on the mailing list, I'd like to
> share some benchmarks I have.  I have a generic BDB/Mnesia compatibility
> layer, but it's fairly slow compared to raw BDB or mnesia access:

Thanks - good stuff.

This does highlight the huge advantages of having an
in-memory, low-impedance, database framework (ets and
mnesia) at your fingertips.

It enables access patterns that would be very impractical,
even with something as fast as BDB.

BR,
Ulf W


From yinso.chen@REDACTED  Sat Oct 13 01:26:34 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 12 Oct 2007 16:26:34 -0700
Subject: [erlang-questions] Parameterized modules
In-Reply-To: <00b201c80d08$32a32030$b029030a@mtl.ubisoft.org>
References: <00b201c80d08$32a32030$b029030a@mtl.ubisoft.org>
Message-ID: <779bf2730710121626r78ec151bxaa7f7df953de68b5@mail.gmail.com>

Hi Denis -

Yes parameterized module is supported but you need a patch in R11B5.  I
asked the same question and you can see it on
http://www.erlang.org/pipermail/erlang-questions/2007-October/029652.htmlc.

Cheers,
yc

On 10/12/07, denis <dloutrein.lists@REDACTED> wrote:
>
>  Hi all,
>
>
>
> I'm interested in using parameterized modules, but as I didn't find other
> documentation except Richard Calsson's paper, I suppose this feature is not
> supported.
>
> Is it safe to use it ? Will it be officially supported in future versions
> ?
>
>
>
> Many thanks
>
> Denis
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071012/7c4d573d/attachment.htm>

From saleyn@REDACTED  Sat Oct 13 03:16:26 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Fri, 12 Oct 2007 20:16:26 -0500
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <2722EBD9-0D75-4655-881D-97FD86DCF7E0@gmail.com>
References: <2722EBD9-0D75-4655-881D-97FD86DCF7E0@gmail.com>
Message-ID: <47101C6A.2020303@gmail.com>

Erlang's SCTP bindings use underlying OS kernel module implementation of 
the protocol.  In Erlang SCTP support uses the same inet driver as TCP 
and UDP.  When you use SCTP you should expect performance similar to TCP 
(at least on Linux and Solaris), however the main advantage of this 
protocol is in its reliability and amount of control that it gives to 
the user application delivered "out-of-band".

Serge

G Bulmer wrote:
> Newbie question.
> 
>> Date: Fri, 12 Oct 2007 12:29:41 +1300
>> From: Bruce Fitzsimons <bruce@REDACTED>
>> Subject: Re: [erlang-questions] EPIPE and TCP sockets
> 
>> ... I'd still like to
>> use the SCTP support in Erlang to demonstrate XMPP over SCTP and see
>> what sort of improvements it might give, ...
> 
> Is the "Erlang SCTP support" likely to give higher message  
> throughput, or is this really about reliability guarantees? Is this  
> OS sensitive, i.e. very much bound to the quality of SCTP  
> implementation for the underlying OS?
> 
> G Bulmer
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 



From yarivsadan@REDACTED  Sat Oct 13 07:59:21 2007
From: yarivsadan@REDACTED (Yariv Sadan)
Date: Fri, 12 Oct 2007 22:59:21 -0700
Subject: [erlang-questions] "Erlang plus BDB: Disrupting the
	Conventional Web Wisdom"
In-Reply-To: <779bf2730710121555l46e944b2j8754301b9aa2a480@mail.gmail.com>
References: <781dd98c0710111007i633fbeecvf89224955ee313e0@mail.gmail.com>
	<31908.207.190.221.98.1192123412.squirrel@geophile.com>
	<470E673C.2090303@munat.com> <47375990.4050504@erlang-consulting.com>
	<470E7EE0.4060703@bitfurnace.com> <470E94C8.6080902@hyber.org>
	<c3c11b6c0710120123j5cbf0f57kf3a3b5736dd587db@mail.gmail.com>
	<470F6222.6050306@hyber.org>
	<17244f480710121503p4432929alb4c8f5260ee42527@mail.gmail.com>
	<779bf2730710121555l46e944b2j8754301b9aa2a480@mail.gmail.com>
Message-ID: <17244f480710122259l21e14b84rfcf39d4ee9d47cb8@mail.gmail.com>

On 10/12/07, YC <yinso.chen@REDACTED> wrote:
>
> On 10/12/07, Yariv Sadan <yarivsadan@REDACTED> wrote:
> > > We can use both Postgres and MySQL from Erlang. With ErlyDB, using
> > > those DBMS's is just as Erlangy as Mnesia for most CRUD and SELECT
> > > operations -- you don't have to write SQL by hand. Many big sites run
> > > on MySQL (YouTube, Flickr, Facebook...), so it definitely scales.
>
> Hi Yariv -
>
> A few questions for ya :)
>
> Is the postgresql driver on par with the mysql driver at this time?  Reading
> from your blog it seems Postgresql is not yet there.

AFAIK, the Postgres driver works. I haven't tested it myself, though.
Other developers have contributed that code -- I just did the MySQL
driver. There's a minor bug in the driver in the current version of
ErlyWeb that will be fixed in the next release.

>
> Would ErlyDB be the DBI/DBD equivalent of the Erlang world?  The nice thing
> about open source is that everyone can do what they want, but having a de
> facto package for something like this is probably essential.

ErlyDB is like ActiveRecord for Rails. It automatically generates
database abstraction code for you from the database metadata. So, if
you create a simple module, e.g. cat.erl:

-module(cat).

and then call erlydb:code_gen() on it, ErlyDB would generate a bunch
of functions that make it easy to work with persistent cats. For
example, you could write the following code

Cat = cat:new_with([{name, "Felix"}]),
Cat1 = cat:save(Cat),          %% executes 'INSERT INTO cat(name)
VALUES ("Felix"); '
"Felix" = cat:name(Cat1),
Cat2 = cat:name(Cat1, "Darwin")
cat:save(Cat2),       %% executes 'UPDATE cat SET name="Darwin" WHERE id=1'
[_] = cat:find({name,'=',"Darwin"}),      % executes 'SELECT FROM cat
WHERE name="Darwin"
cat:delete(Cat2)      % executes 'DELETE FROM cat WHERE id=1

You can also do more advanced things, like one-to-many and
many-to-many relations. (In the Vimagi codebase, you won't find a
single line of SQL :) )

I can't say if it's a de-facto package -- that depends on how many
people want to adopt it.

>
> > > In Vimagi, I use MySQL for persistent data and Mnesia for live session
> > > data. It works great, even on a RAM-constrained VPS.
>
> Is the live session data a cached data or actually the session table? I
> think Mnesia would be great as a cache server (e.g. memcached), but haven't
> thought through it, and if you have done so, would love to hear your take.

I think one of the best things about building webapps with Erlang is
Mnesia. You can keep your session state in Mnesia without having to
put it in your database (slow), on disk (also slow) or in memcached
(requires more moving parts).  I currently use Mnesia just for session
state -- I don't do caching in Vimagi yet. I'm sure that implementing
caching with Mnesia would be just as easy -- I just haven't tried it.
I don't know enough about memcached to say which would work best under
very high loads, but Ericsson's experience with Mnesia indicates that
Mnesia can support more than AJAXy blog engines :)

Mnesia is shared memory done right. It's easy to work with, it can
store any Erlang data, it's transactional, and due to Erlang's hot
code swapping you don't lose your in-RAM data when you do code
upgrades. Without hot code swapping, keeping seesion state purely in
RAM would be a bad choice because you'd lose it when having to restart
the server to do code upgrades. (To do code upgrades in Vimagi, I ssh
into the box and execute "svn up" :) ).

But don't use Mnesia for high-volume persistent storage -- use MySQL,
Postgres or BDB (or create your own driver for Oracle or whatnot).

Cheers,
Yariv


From bjorn@REDACTED  Sat Oct 13 08:02:33 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 13 Oct 2007 08:02:33 +0200
Subject: [erlang-questions] Learning erlang - is this an efficient way
	to build a list?
In-Reply-To: <779bf2730710121357s45e5d08ewd96d4d0a49397b59@mail.gmail.com>
References: <12958075.post@talk.nabble.com>
	<779bf2730710121357s45e5d08ewd96d4d0a49397b59@mail.gmail.com>
Message-ID: <o1d4vjtuva.fsf@erix.ericsson.se>

YC <yinso.chen@REDACTED> writes:

> In general what I've seen is to use the cons operator rather than append,
> because append is O(n) and cons is O(1), but in your case it's fine as you
> are still reversing the list, and hence your append is O(1) as well.  I
> guess the key is in reversing the list ;)

Yes, the important thing is to build the list in reverse order. '++' is fine
as long as the accumulator argument is to the right.

In this simple case, the compiler will itself rewrite

[spawn(fun ring:recv/0)] ++ L

to

[spawn(fun ring:recv/0)|L]

so the cost is exactly the same.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From Bruce@REDACTED  Sat Oct 13 11:45:18 2007
From: Bruce@REDACTED (Bruce Fitzsimons)
Date: Sat, 13 Oct 2007 22:45:18 +1300
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <2722EBD9-0D75-4655-881D-97FD86DCF7E0@gmail.com>
References: <2722EBD9-0D75-4655-881D-97FD86DCF7E0@gmail.com>
Message-ID: <471093AE.3060600@Fitzsimons.org>

G Bulmer wrote:
> Newbie question.
>
>   
>> Date: Fri, 12 Oct 2007 12:29:41 +1300
>> From: Bruce Fitzsimons <bruce@REDACTED>
>> Subject: Re: [erlang-questions] EPIPE and TCP sockets
>>     
>
>   
>> ... I'd still like to
>> use the SCTP support in Erlang to demonstrate XMPP over SCTP and see
>> what sort of improvements it might give, ...
>>     
>
> Is the "Erlang SCTP support" likely to give higher message  
> throughput, or is this really about reliability guarantees? Is this  
> OS sensitive, i.e. very much bound to the quality of SCTP  
> implementation for the underlying OS?
>
>   
Throughput is unlikely to be a benefit; considering we have many years 
experience in TCP behaviour and tuning and not much at all with SCTP. 
SCTP is a more complex protocol, but does avoid several TCP corner cases 
including head of line blocking 
http://en.wikipedia.org/wiki/Head-of-line_blocking.

Given that it is most used in friendly environments, I think we are 
still to see what new DoS attacks SCTP could enable, but the TCP ones 
are fixed and thought was given to preventing new ones.

I am most interested in SCTP to take advantage of the native connection 
heart-beating, something commonly implemented in the application layer 
but not present as standard in XMPP. With the "standard" TCP connection 
timeout at 2 hours the TX buffer can be filling on the sending host 
without the application being aware that they are not being delivered, 
and there is no way to tell what quantity of data was reliably delivered 
when the connection is finally terminated (Tony's problem). This 
manifests in XMPP as messages being silently blackholed and presence 
information that is way out of date. This only occurs when the recipient 
disappears rather than terminates the TCP connection but this is fairly 
common on the Internet. (I'm quite likely to be corrected here, in my 
defense my Stevens is at work and the failure cases are subtle.) Most 
XMPP implementations reduce the timeout length but this can never be a 
complete solution.

Regards,
Bruce


From per@REDACTED  Sat Oct 13 14:36:27 2007
From: per@REDACTED (Per Hedeland)
Date: Sat, 13 Oct 2007 14:36:27 +0200 (CEST)
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <471093AE.3060600@Fitzsimons.org>
Message-ID: <200710131236.l9DCaQNN093757@pluto.hedeland.org>

Bruce Fitzsimons <Bruce@REDACTED> wrote:
>>   
>Throughput is unlikely to be a benefit; considering we have many years 
>experience in TCP behaviour and tuning and not much at all with SCTP. 
>SCTP is a more complex protocol, but does avoid several TCP corner cases 
>including head of line blocking 
>http://en.wikipedia.org/wiki/Head-of-line_blocking.

Hm, I can't quite see how that relates to TCP vs SCTP. Presumably you're
not actually referring to the situation in a network switch (i.e. layer
2 device), since the upper-layer protocol won't make a difference there?
I can see how this phenomenon could occur with "naive" multiplexing of
multiple "channels" on a single TCP session, like, um, the Erlang
distribution does (though I can't see it happening with Erlang
distribution since the "channels", i.e. inter-process messages, won't be
blocked at the receiving end) - is that the connection?

Of course this can be handled by per-channel flow control (like e.g. SSH
does) when using TCP, but I guess this user-level complexity can be
avoided by using SCTP instead.

>I am most interested in SCTP to take advantage of the native connection 
>heart-beating, something commonly implemented in the application layer 
>but not present as standard in XMPP. With the "standard" TCP connection 
>timeout at 2 hours the TX buffer can be filling on the sending host 
>without the application being aware that they are not being delivered, 

The 2 hours would be for the (misnamed) keep-alive functionality in TCP
- if there actually is output data waiting to be ack'ed, the "standard"
timeout before the connection is declared dead is generally much
shorter, on the order of 5 minutes or so - though of course that can
still be "way too long" in some cases.

>and there is no way to tell what quantity of data was reliably delivered 
>when the connection is finally terminated (Tony's problem).

Yes, but that's the same story as we've frequently discussed here
regarding whether Erlang distribution is "reliable" - if you *really*
want *reliable*, knowing that the data was successfully received by the
network stack at the destination is only the first chapter in that
story...

--Per Hedeland


From saleyn@REDACTED  Sat Oct 13 16:01:43 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Sat, 13 Oct 2007 09:01:43 -0500
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <200710131236.l9DCaQNN093757@pluto.hedeland.org>
References: <200710131236.l9DCaQNN093757@pluto.hedeland.org>
Message-ID: <4710CFC7.1040305@gmail.com>

Per Hedeland wrote:
> Bruce Fitzsimons <Bruce@REDACTED> wrote:
>>>   
>> Throughput is unlikely to be a benefit; considering we have many years 
>> experience in TCP behaviour and tuning and not much at all with SCTP. 
>> SCTP is a more complex protocol, but does avoid several TCP corner cases 
>> including head of line blocking 
>> http://en.wikipedia.org/wiki/Head-of-line_blocking.
> 
> Hm, I can't quite see how that relates to TCP vs SCTP. Presumably you're
> not actually referring to the situation in a network switch (i.e. layer
> 2 device), since the upper-layer protocol won't make a difference there?
> I can see how this phenomenon could occur with "naive" multiplexing of
> multiple "channels" on a single TCP session, like, um, the Erlang
> distribution does (though I can't see it happening with Erlang
> distribution since the "channels", i.e. inter-process messages, won't be
> blocked at the receiving end) - is that the connection?

I also think that Head-of-line blocking (HOLB) wiki description is not 
very representative in regards to TCP/SCTP.  The HOL means that if you 
have consecutive packets 1,2,3 sent from A to B, and packet 3 is lost, 
then B holds 1 and 2 until it gets 3 retransmitted, so that it would 
preserve the byte ordering.  In SCTP this may not be a problem because 
packet 3 could belong to a different stream.

> Of course this can be handled by per-channel flow control (like e.g. SSH
> does) when using TCP, but I guess this user-level complexity can be
> avoided by using SCTP instead.

Hmm, will the session-level per-channel flow control resolve the issue 
above?  At the transport (TCP) layer it would still have to guarantee 
the proper byte ordering, therefore forcing HOLB.

Serge


From gbulmer@REDACTED  Sat Oct 13 17:13:52 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sat, 13 Oct 2007 16:13:52 +0100
Subject: [erlang-questions] Learning erlang - is this an efficient way to
	build a list?
Message-ID: <F5AFC338-562E-4B97-AEB8-C69F92E0A1BA@gmail.com>

An alternative is just
[spawn(fun ring:recv/0) || lists:seq(2,10)]

> Date: Fri, 12 Oct 2007 12:13:48 -0700 (PDT)
> From: graemebe <graemebe@REDACTED>
> Subject: [erlang-questions] Learning erlang - is this an efficient way
> 	to build a list?
> ...
> % Spawn some processes and send a message to each.
> %
> start() ->
> 	M=10,
> 	P=ring:spawnem(M),
> 	[X ! "hello world!" || X <- P].
>
> % Function to Spawn N processes
> %
> spawnem(N) when is_integer(N), N > 0 -> spawnem_acc(N, [spawn(fun
> ring:recv/0)]).
> spawnem_acc(N, L) when is_integer(N), N > 1 -> spawnem_acc(N-1,  
> [spawn(fun
> ring:recv/0)] ++ L);
> spawnem_acc(1, L) -> L.
>

As it stands, an alternative is to replace spawnem and spawnem_acc  
with a list comprehension:

start() ->
	M=10,
	P=[spawn(fun ring:recv/0) || _N <- lists:seq(2,10)]),  %%  <<---  
changed
	[X ! "hello world!" || X <- P].

and delete spawnem, and spawnem_acc.

GB



From gbulmer@REDACTED  Sat Oct 13 17:40:59 2007
From: gbulmer@REDACTED (G Bulmer)
Date: Sat, 13 Oct 2007 16:40:59 +0100
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
Message-ID: <CBE0F18C-B44E-49FA-B43D-093A78CB2DF8@gmail.com>

Bruce

Thank you, that was a very helpful answer.

I understand the 'advertising' feature list for SCTP, but I've never  
appreciated the issue you explained.
I'd only skimmed http://tools.ietf.org/html/rfc3286, and I'd not  
noticed anything about 'heart-beating' until now either (9.1 and  
9.2e), so that was great!

I strongly agree that it's a bit early to deploy SCTP in an hostile  
environment, unless there are *critical* benefits that outweigh the  
risks (and you can manage the risks :-).

Anybody - On a related track, do all Erlang messages between two  
specific Erlang nodes go down one socket?
[I thought the response to my question would be out-of-order delivery  
being the killer SCTP feature, but maybe that isn't an issue].

Thanks again
Garry Bulmer

> Date: Sat, 13 Oct 2007 22:45:18 +1300
> From: Bruce Fitzsimons <Bruce@REDACTED>
> Subject: Re: [erlang-questions] SCTP support (was EPIPE and TCP
> 	sockets)
> To: G Bulmer <gbulmer@REDACTED>
> Cc: "Erlang-Questions \(E-mail\)" <erlang-questions@REDACTED>
> Message-ID: <471093AE.3060600@REDACTED>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> G Bulmer wrote:
>
>> Newbie question.
>>
>>
>>
>>> Date: Fri, 12 Oct 2007 12:29:41 +1300
>>> From: Bruce Fitzsimons <bruce@REDACTED>
>>> Subject: Re: [erlang-questions] EPIPE and TCP sockets
>>>
>>>
>>
>>
>>
>>> ... I'd still like to
>>> use the SCTP support in Erlang to demonstrate XMPP over SCTP and see
>>> what sort of improvements it might give, ...
>>>
>>>
>>
>> Is the "Erlang SCTP support" likely to give higher message
>> throughput, or is this really about reliability guarantees? Is this
>> OS sensitive, i.e. very much bound to the quality of SCTP
>> implementation for the underlying OS?
>>
>>
>>
> Throughput is unlikely to be a benefit; considering we have many years
> experience in TCP behaviour and tuning and not much at all with SCTP.
> SCTP is a more complex protocol, but does avoid several TCP corner  
> cases
> including head of line blocking
> http://en.wikipedia.org/wiki/Head-of-line_blocking.
>
> Given that it is most used in friendly environments, I think we are
> still to see what new DoS attacks SCTP could enable, but the TCP ones
> are fixed and thought was given to preventing new ones.
>
> I am most interested in SCTP to take advantage of the native  
> connection
> heart-beating, something commonly implemented in the application layer
> but not present as standard in XMPP. With the "standard" TCP  
> connection
> timeout at 2 hours the TX buffer can be filling on the sending host
> without the application being aware that they are not being delivered,
> and there is no way to tell what quantity of data was reliably  
> delivered
> when the connection is finally terminated (Tony's problem). This
> manifests in XMPP as messages being silently blackholed and presence
> information that is way out of date. This only occurs when the  
> recipient
> disappears rather than terminates the TCP connection but this is  
> fairly
> common on the Internet. (I'm quite likely to be corrected here, in my
> defense my Stevens is at work and the failure cases are subtle.) Most
> XMPP implementations reduce the timeout length but this can never be a
> complete solution.
>
> Regards,
> Bruce
>


From joelr1@REDACTED  Sat Oct 13 18:20:14 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Sat, 13 Oct 2007 17:20:14 +0100
Subject: [erlang-questions] Erlang and the Amazon Elastic Computing Cloud
Message-ID: <674E39BE-4FF0-45AB-8A66-CC361FE15711@gmail.com>

Folks,

I just posted a summary of a project I recently completed for a  
startup. The focus is on using a boot server to manage code updates  
within a cluster made of EC2 instances.

http://wagerlabs.com/archives/121.html

Please let me know if you have any comments and if I should expand  
upon any portion of the article.

	Thanks, Joel

--
http://wagerlabs.com







From fredrik.svahn@REDACTED  Sat Oct 13 18:41:41 2007
From: fredrik.svahn@REDACTED (Fredrik Svahn)
Date: Sat, 13 Oct 2007 18:41:41 +0200
Subject: [erlang-questions] Searching binaries with matching?
Message-ID: <ce37fb00710130941w251f3f9eua493823fbf96779c@mail.gmail.com>

Hi,

I would find it extremely useful to have a binary matching which searches
for the first occurrence of a supplied pattern in a binary, i.e. to extract
the body from an HTML file I could write something like:

<< _/binary, "<body>", Body/binary, "</body>", _/binary >> = MyHTMLFile.

or to split e.g. a SIP message into lines:

get_sipheaders(<<>>, Acc) -> Acc.
get_sipheaders(<<Line/binary, "\r\n", Rest/binary>>, Acc) ->
  get_sipheaders(Rest, [Line | Acc]).

get_sipheaders(<<"line1\r\nline2\r\nline3\r\n">>,[]) would result in
[<<"line3">>, <<"line2">>, <<"line1">>]

or a simple grep:

grep(SearchPattern, File) ->
  {ok, FileBin} = file:read_file(File),
  try <<_/binary, SearchPattern/binary, _/binary>> = FileBin of
     _Match -> match
  catch
    error:{badmatch, _} -> nomatch
  end.

I know that it would be possible to rewrite it and recursively search
through a binary step by step by specifying the size of the first binary,
but IIRC that is not very efficient compared to most modern search
algorithms (for comparison, have a look at e.g.
http://ridiculousfish.com/blog/archives/2006/05/30/old-age-and-treachery/).
I even believe someone stated that it was usually more efficient to convert
the binary to a list before searching it.

My questions to the mailing list would be:
1. Is it possible to do something like this today (or in a near future)
efficiently, e.g. through the regexp library?
2. Is there anything in the syntax above which would make it improper or
even impossible to add a "searching match" to binary matches (given enough
time and resources)?
3. If so, would it be possible to achieve the same thing with a slightly
improved syntax?

BR /Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071013/052c54c1/attachment.htm>

From sean.hinde@REDACTED  Sat Oct 13 18:53:26 2007
From: sean.hinde@REDACTED (Sean Hinde)
Date: Sat, 13 Oct 2007 17:53:26 +0100
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <CBE0F18C-B44E-49FA-B43D-093A78CB2DF8@gmail.com>
References: <CBE0F18C-B44E-49FA-B43D-093A78CB2DF8@gmail.com>
Message-ID: <DCF55BD9-644E-4C8C-B690-7C4AD6685DFF@gmail.com>


On 13 Oct 2007, at 16:40, G Bulmer wrote:

>
>
> Anybody - On a related track, do all Erlang messages between two
> specific Erlang nodes go down one socket?

Yes.

> [I thought the response to my question would be out-of-order delivery
> being the killer SCTP feature, but maybe that isn't an issue].

It can issue if you are not aware of it. If you send a large message  
via the normal erlang distribution then all other messages between  
nodes will be blocked until the large message has been sent. And,  
yes, it is a real example..

sctp could provide an elegant solution to this - assign a separate  
sctp stream to each pair of processes that are exchanging messages.

Another answer would be for the existing tcp based erlang  
distribution mechanism to split up large erlang messages after  
conversion to the external protocol byte stream and multiplex at a  
lower level.

I suspect that either of these mechanisms would reveal bugs in  
previously working code, though neither should break the written  
guarantees about message ordering.

Sean




From graemebe@REDACTED  Sat Oct 13 18:58:26 2007
From: graemebe@REDACTED (graemebe)
Date: Sat, 13 Oct 2007 09:58:26 -0700 (PDT)
Subject: [erlang-questions] Learning erlang - is this an efficient way
 to build a list?
In-Reply-To: <o1d4vjtuva.fsf@erix.ericsson.se>
References: <12958075.post@talk.nabble.com>
	<779bf2730710121357s45e5d08ewd96d4d0a49397b59@mail.gmail.com>
	<o1d4vjtuva.fsf@erix.ericsson.se>
Message-ID: <13191513.post@talk.nabble.com>


Thanks for the responses everyone. I was trying to build the list in reverse
order after reading that mentioned in Joe Armstrong's book. I did try the
X|L syntax at one point but for some reason I got a weird bug and abandoned
it. Have got it working now. I also like the other idea from GB for doing it
with lists:seq, which takes less code - one of the things I like about
erlang is that you can write cool functions in only a few chars.

>Yes, the important thing is to build the list in reverse order. '++' is
fine
>as long as the accumulator argument is to the right.
>
>In this simple case, the compiler will itself rewrite
>
>[spawn(fun ring:recv/0)] ++ L
>
>to
>
>[spawn(fun ring:recv/0)|L]
>
>so the cost is exactly the same.

-- 
View this message in context: http://www.nabble.com/Learning-erlang---is-this-an-efficient-way-to-build-a-list--tf4540288.html#a13191513
Sent from the Erlang Questions mailing list archive at Nabble.com.



From rsaccon@REDACTED  Sat Oct 13 19:04:02 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Sat, 13 Oct 2007 14:04:02 -0300
Subject: [erlang-questions] Erlang and the Amazon Elastic Computing Cloud
In-Reply-To: <674E39BE-4FF0-45AB-8A66-CC361FE15711@gmail.com>
References: <674E39BE-4FF0-45AB-8A66-CC361FE15711@gmail.com>
Message-ID: <eb4177b00710131004w6751a338gb494e53b62cc318f@mail.gmail.com>

Great article ! Have you found a way around the DHCP initialization
issue ("operating system high resolution timer"  error) when starting
the Erlang VM at OS startup ? I observed that error, which always
resulted in a coredump  on all OS I tried on EC2. To prevent it I just
while-loop in a shell script  trying to start the VM, with some sleep
between the attempts.

btw. for anybody interested in Erlang on EC2, I have a script to build
and configure an OS image for EC2, need to sort out some minor issues
before releasing it. But for regular application updates the approach
as described by Joel is better suited, because it takes just too long
to build an OS image, even automated.

-- 
Roberto Saccon
http://rsaccon.com


From joelr1@REDACTED  Sat Oct 13 19:11:31 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Sat, 13 Oct 2007 18:11:31 +0100
Subject: [erlang-questions] Erlang and the Amazon Elastic Computing Cloud
In-Reply-To: <eb4177b00710131004w6751a338gb494e53b62cc318f@mail.gmail.com>
References: <674E39BE-4FF0-45AB-8A66-CC361FE15711@gmail.com>
	<eb4177b00710131004w6751a338gb494e53b62cc318f@mail.gmail.com>
Message-ID: <EFAD9A47-EE89-49BE-88B3-1FFD4C9809F1@gmail.com>


On Oct 13, 2007, at 6:04 PM, Roberto Saccon wrote:

> Great article !

Thanks!

> Have you found a way around the DHCP initialization
> issue ("operating system high resolution timer"  error) when starting
> the Erlang VM at OS startup ?

I believe it only happens on Ubuntu Feisty. I think someone told me  
this. We haven't found a workaround, though.

What OS do you run?

> But for regular application updates the approach
> as described by Joel is better suited, because it takes just too long
> to build an OS image, even automated.

I have yet to write the more interesting article :-). It will begin  
like this...

"Bowing to the OTP application packaging procedure, I wanted to have  
a way of upgrading applications with a "push of a button". More  
precisely, I wanted to be able to type make:all() to rebuild my  
application and then type sync:all() to push updated modules to all  
nodes in my cluster."

Yes, that also involves Amazon and EC2 but is applicable to any other  
scenario.

	Joel

--
http://wagerlabs.com



From rsaccon@REDACTED  Sat Oct 13 19:20:24 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Sat, 13 Oct 2007 14:20:24 -0300
Subject: [erlang-questions] Erlang and the Amazon Elastic Computing Cloud
In-Reply-To: <EFAD9A47-EE89-49BE-88B3-1FFD4C9809F1@gmail.com>
References: <674E39BE-4FF0-45AB-8A66-CC361FE15711@gmail.com>
	<eb4177b00710131004w6751a338gb494e53b62cc318f@mail.gmail.com>
	<EFAD9A47-EE89-49BE-88B3-1FFD4C9809F1@gmail.com>
Message-ID: <eb4177b00710131020v11139f81m7eded6ba48554fd2@mail.gmail.com>

On 10/13/07, Joel Reymont <joelr1@REDACTED> wrote:
>
> On Oct 13, 2007, at 6:04 PM, Roberto Saccon wrote:
>
> > Great article !
>
> Thanks!
>
> > Have you found a way around the DHCP initialization
> > issue ("operating system high resolution timer"  error) when starting
> > the Erlang VM at OS startup ?
>
> I believe it only happens on Ubuntu Feisty. I think someone told me
> this. We haven't found a workaround, though.
>
> What OS do you run?
>

I see. They were all Debian or Debian based ...

> > But for regular application updates the approach
> > as described by Joel is better suited, because it takes just too long
> > to build an OS image, even automated.
>
> I have yet to write the more interesting article :-). It will begin
> like this...
>
> "Bowing to the OTP application packaging procedure, I wanted to have
> a way of upgrading applications with a "push of a button". More
> precisely, I wanted to be able to type make:all() to rebuild my
> application and then type sync:all() to push updated modules to all
> nodes in my cluster."
>
> Yes, that also involves Amazon and EC2 but is applicable to any other
> scenario.
>
>         Joel
>
> --
> http://wagerlabs.com
>
>


-- 
Roberto Saccon
http://rsaccon.com


From per@REDACTED  Sat Oct 13 23:53:51 2007
From: per@REDACTED (Per Hedeland)
Date: Sat, 13 Oct 2007 23:53:51 +0200 (CEST)
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <4710CFC7.1040305@gmail.com>
Message-ID: <200710132153.l9DLrppC005989@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>I also think that Head-of-line blocking (HOLB) wiki description is not 
>very representative in regards to TCP/SCTP.  The HOL means that if you 
>have consecutive packets 1,2,3 sent from A to B, and packet 3 is lost, 
>then B holds 1 and 2 until it gets 3 retransmitted, so that it would 
>preserve the byte ordering.  In SCTP this may not be a problem because 
>packet 3 could belong to a different stream.

OK - but if you have significant packet loss, you should try to fix that
instead.:-)

>> Of course this can be handled by per-channel flow control (like e.g. SSH
>> does) when using TCP, but I guess this user-level complexity can be
>> avoided by using SCTP instead.
>
>Hmm, will the session-level per-channel flow control resolve the issue 
>above?  At the transport (TCP) layer it would still have to guarantee 
>the proper byte ordering, therefore forcing HOLB.

Right, it doesn't solve that - it deals with the (probably common) case
that some channel receivers are slower than others. If you're not
prepared to do unlimited buffering in the receiving de-multiplexer (like
Erlang distribution does:-), the slowest receiver sets the limit for the
throughput (possibly at zero) if you can't do per-channel flow control.

--Per



From fredrik.svahn@REDACTED  Sun Oct 14 01:18:38 2007
From: fredrik.svahn@REDACTED (Fredrik Svahn)
Date: Sun, 14 Oct 2007 01:18:38 +0200
Subject: [erlang-questions] Searching binaries with matching?
In-Reply-To: <ce37fb00710130941w251f3f9eua493823fbf96779c@mail.gmail.com>
References: <ce37fb00710130941w251f3f9eua493823fbf96779c@mail.gmail.com>
Message-ID: <ce37fb00710131618j7358426dt327b81a7568dba87@mail.gmail.com>

By the way, although I think I understand why I get the following badmatch,
I find it slightly counter-intuitive:

49> B="hello".
"hello"
50> << "hello", D/binary>> = <<"hello world!">>.
<<"hello world!">>
51> << B, E/binary>> = <<"hello world!">>.
** exited: {{badmatch,<<"hello world!">>},[{erl_eval,expr,3}]} **
=ERROR REPORT==== 14-Oct-2007::00:50:51 ===
Error in process <0.141.0> with exit value: {{badmatch,<<12
bytes>>},[{erl_eval,expr,3}]}

BR /Fredrik


On 10/13/07, Fredrik Svahn <fredrik.svahn@REDACTED> wrote:
>
> Hi,
>
> I would find it extremely useful to have a binary matching which searches
> for the first occurrence of a supplied pattern in a binary, i.e. to
> extract the body from an HTML file I could write something like:
>
> << _/binary, "<body>", Body/binary, "</body>", _/binary >> = MyHTMLFile.
>
> or to split e.g. a SIP message into lines:
>
> get_sipheaders(<<>>, Acc) -> Acc.
> get_sipheaders(<<Line/binary, "\r\n", Rest/binary>>, Acc) ->
>   get_sipheaders(Rest, [Line | Acc]).
>
> get_sipheaders(<<"line1\r\nline2\r\nline3\r\n">>,[]) would result in
> [<<"line3">>, <<"line2">>, <<"line1">>]
>
> or a simple grep:
>
> grep(SearchPattern, File) ->
>   {ok, FileBin} = file:read_file(File),
>   try <<_/binary, SearchPattern/binary, _/binary>> = FileBin of
>      _Match -> match
>   catch
>     error:{badmatch, _} -> nomatch
>   end.
>
> I know that it would be possible to rewrite it and recursively search
> through a binary step by step by specifying the size of the first binary,
> but IIRC that is not very efficient compared to most modern search
> algorithms (for comparison, have a look at e.g.
> http://ridiculousfish.com/blog/archives/2006/05/30/old-age-and-treachery/).
> I even believe someone stated that it was usually more efficient to convert
> the binary to a list before searching it.
>
> My questions to the mailing list would be:
> 1. Is it possible to do something like this today (or in a near future)
> efficiently, e.g. through the regexp library?
> 2. Is there anything in the syntax above which would make it improper or
> even impossible to add a "searching match" to binary matches (given enough
> time and resources)?
> 3. If so, would it be possible to achieve the same thing with a slightly
> improved syntax?
>
> BR /Fredrik
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071014/c78599f7/attachment.htm>

From joelr1@REDACTED  Sun Oct 14 16:47:55 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Sun, 14 Oct 2007 15:47:55 +0100
Subject: [erlang-questions] Upgrading Erlang clusters for fun and profit
Message-ID: <DEC8D552-17E6-424B-9BF8-613B5DB24B9E@gmail.com>

Find yourself hating the standard Erlang/OTP upgrade procedure?

Ever wanted to upgrade your whole cluster with a push of a button?

http://wagerlabs.com/archives/125.html

This is not for the faint-hearted!

	Cheers, Joel


--
http://wagerlabs.com







From berlin.brown@REDACTED  Mon Oct 15 00:26:43 2007
From: berlin.brown@REDACTED (Berlin Brown)
Date: Sun, 14 Oct 2007 18:26:43 -0400
Subject: [erlang-questions] Can't get yaws to recognize modules
Message-ID: <df27a55b0710141526j1c38e074v8fc1f99cca8d39e2@mail.gmail.com>

I have a simple yaws application and can't figure why yaws won't
recognize functions in my module. (yaws 1.73)

Also, I dont really understand the role of the *.app file in the ebin directory.

Here is my app file which I am assuming is loaded automatically:
{application, ebotlist_app,
 [{description, "Ebotlist"},
  {vsn, "0.1"},
  {modules, [ botlist_find_links ] },
  {applications, [kernel, stdlib]},
  {env,
   [ {log_prefix, "ebotlist_log"} ] },
  {mod, {ebotlist_app,[]}} ]
}.

This is all that is in the source, beam file and that is found in the
ebin directory.

-export([find_links/0]).
-record(entity_links, {main_url, url_title, keywords, rating,
full_name, created_on}).

find_links() ->
    F = fun() ->
		WP = mnesia:table_info(entity_links, wild_pattern),
		[EntityLink || EntityLink <- mnesia:match_object(WP)]
        end,
    case mnesia:transaction(F) of
        {atomic, Result}  -> {ok, Result};
        {aborted, Reason} -> {error, Reason}
    end.

The error itself is displayed by yaws:

function find_links/0 undefined

You can see most of the code here:
http://octaned.googlecode.com/svn/trunk/octaned/apps/ebotlist/

-- 
Berlin Brown
http://botspiritcompany.com/botlist/spring/help/about.html
newspirit technologies


From bbmaj7@REDACTED  Mon Oct 15 08:28:08 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Mon, 15 Oct 2007 16:28:08 +1000 (EST)
Subject: [erlang-questions] Upgrading Erlang clusters for fun and profit
In-Reply-To: <DEC8D552-17E6-424B-9BF8-613B5DB24B9E@gmail.com>
Message-ID: <993616.89906.qm@web52002.mail.re2.yahoo.com>


--- Joel Reymont <joelr1@REDACTED> wrote:

> http://wagerlabs.com/archives/125.html
> 
> This is not for the faint-hearted!

There was no mention of where code_change() fits into the process.

I immediately thought that a two-phase commit would be useful: phase 1 to
suspend and test if the upgrade is supported; and if all nodes say ok, then
perform phase 2 to load the new code on all nodes. If any report a problem then
resume on all nodes without changing code.

But OTP doesn't seem to have any way to pre-test if the code_change callback
implementation can support the current and desired versions.



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From joelr1@REDACTED  Mon Oct 15 10:40:00 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Mon, 15 Oct 2007 09:40:00 +0100
Subject: [erlang-questions] Upgrading Erlang clusters for fun and profit
In-Reply-To: <993616.89906.qm@web52002.mail.re2.yahoo.com>
References: <993616.89906.qm@web52002.mail.re2.yahoo.com>
Message-ID: <61E1ABC7-AF37-4677-BF83-8581BFB7F3B0@gmail.com>


On Oct 15, 2007, at 7:28 AM, Richard Andrews wrote:

> There was no mention of where code_change() fits into the process.

It doesn't. It's not necessary when upgrading app-by-app and not all  
the processes suspended and restarted would be gen_servers.

> I immediately thought that a two-phase commit would be useful:  
> phase 1 to
> suspend and test if the upgrade is supported; and if all nodes say  
> ok, then
> perform phase 2 to load the new code on all nodes.

It's better to do a rolling upgrade, I think, node by node. It avoids  
taking your whole cluster out of service.


--
http://wagerlabs.com







From tonyg@REDACTED  Mon Oct 15 12:03:42 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Mon, 15 Oct 2007 11:03:42 +0100
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <200710112244.l9BMiHCt012626@pluto.hedeland.org>
References: <200710112244.l9BMiHCt012626@pluto.hedeland.org>
Message-ID: <47133AFE.1070700@lshift.net>

Per Hedeland wrote:
> Hm, I don't think that's correct - AFAIK SIGPIPE/EPIPE happens only if
> you try to write more data to a socket when the other end is considered
> to be closed (either due to receiving FIN or RST, or due to
> retransmission timeout).
> 
> Closing the socket with unack'ed data normally doesn't do anything in
> particular, the close() returns immediately but the kernel holds on to
> the data and tries to (re)transmit it according to the standard TCP
> logic, and you won't be told if it fails.

Thanks, Per. That's useful. It tells me that TCP itself can do what I
want, but the sockets interface isn't set up for it. I'll do something
different at the application level.

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg@REDACTED


From tonyg@REDACTED  Mon Oct 15 12:06:47 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Mon, 15 Oct 2007 11:06:47 +0100
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <470EB1E5.4010100@fitzsimons.org>
References: <470E847C.7090409@lshift.net> <470EB1E5.4010100@fitzsimons.org>
Message-ID: <47133BB7.50607@lshift.net>

Bruce Fitzsimons wrote:
> I saw the same sort of issue in a rant against the jabber/XMPP protocol;
> it treats TCP as "reliable" in an application to application sense. TCP
> isn't, although 99% of the time it is probably close enough.

Indeed. I'm investigating this as part of thinking about "reliable
delivery" (in all its manifold senses) in AMQP.

> I'd still like to
> use the SCTP support in Erlang to demonstrate XMPP over SCTP and see
> what sort of improvements it might give

The next rev of AMQP will support SCTP as a transport for the protocol.

> There is a way using getsockopt to find out if there is a tcp buffer of
> unsent/un-acked data, but I believe it is *linux-specific*

Interesting! Thanks. The platform-specificity rules it out for this
application, but it's good to know for future reference.

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg@REDACTED


From Bruce@REDACTED  Mon Oct 15 12:54:57 2007
From: Bruce@REDACTED (Bruce Fitzsimons)
Date: Mon, 15 Oct 2007 23:54:57 +1300
Subject: [erlang-questions] SCTP support (was EPIPE and TCP sockets)
In-Reply-To: <200710131236.l9DCaQNN093757@pluto.hedeland.org>
References: <200710131236.l9DCaQNN093757@pluto.hedeland.org>
Message-ID: <47134701.4060106@Fitzsimons.org>

Per Hedeland wrote:
> Bruce Fitzsimons <Bruce@REDACTED> wrote:
>   
>> http://en.wikipedia.org/wiki/Head-of-line_blocking.
>>     
>
> Hm, I can't quite see how that relates to TCP vs SCTP. Presumably you're
> not actually referring to the situation in a network switch (i.e. layer
> 2 device), since the upper-layer protocol won't make a difference there?
> I can see how this phenomenon could occur with "naive" multiplexing of
> multiple "channels" on a single TCP session, like, um, the Erlang
> distribution does (though I can't see it happening with Erlang
> distribution since the "channels", i.e. inter-process messages, won't be
> blocked at the receiving end) - is that the connection?
>
> Of course this can be handled by per-channel flow control (like e.g. SSH
> does) when using TCP, but I guess this user-level complexity can be
> avoided by using SCTP instead.
>   
I agree. I wasn't specifically saying that head-of-line blocking 
impacted XMPP, just that SCTP can use streams to avoid it rather than 
using yet another socket (a limited resource for a server) and managing 
them at the app level. I've seen enough buggy implementations of 
application layer flow control, keep-alives etc.

The idea with SCTP is not that the application layer can't do these 
things using existing protocols but that one decent implementation at a 
lower layer can make it easier on everyone. Back in the real world 
though I have not seen a real-world use case of SCTP streams (e.g >1 
stream per association) or multihoming yet as the applications that I've 
seen using it (SIGTRAN) have all this complexity in the upper layers at 
least once :-) I'm looking forward to playing with DIAMETER(/DCCA) to 
see if it is any different (SCTP support is mandated I believe).
> The 2 hours would be for the (misnamed) keep-alive functionality in TCP
> - if there actually is output data waiting to be ack'ed, the "standard"
> timeout before the connection is declared dead is generally much
> shorter, on the order of 5 minutes or so - though of course that can
> still be "way too long" in some cases.
>
>   
Yes you are right. The particular corner case that impacts XMPP with the 
keep-alives is that it uses the TCP connection status (server<->client) 
as an application layer indicator that the client is still there. So the 
last-advised state ("available to chat") of a dead client stays visible 
to all parties for quite some time if noone send the dead one any 
messages. Sending application-layer keep-alives helps, but the last XMPP 
extension proposal I saw just did it one-way and relied on the TCP 
timeout. It works but I'm used to sub-second notification that links are 
out. All the timers can be shortened but they're still "way too long" 
for me.
>> and there is no way to tell what quantity of data was reliably delivered 
>> when the connection is finally terminated (Tony's problem).
>>     
>
> Yes, but that's the same story as we've frequently discussed here
> regarding whether Erlang distribution is "reliable" - if you *really*
> want *reliable*, knowing that the data was successfully received by the
> network stack at the destination is only the first chapter in that
> story...
>   
I agree. UDP can be just as "reliable" :-)

I think the discussion with Sean about putting Erlang distribution over 
SCTP is interesting, the multihoming might be useful for some 
applications where the node mesh needs to be maintained over a distance 
(or the internet).

Cheers,
Bruce


From hans.bolinder@REDACTED  Mon Oct 15 14:59:05 2007
From: hans.bolinder@REDACTED (Hans Bolinder)
Date: Mon, 15 Oct 2007 14:59:05 +0200
Subject: [erlang-questions] Test Coverage -- proposed patch in erl_pp
In-Reply-To: <46E2D36D.4070509@ituniv.se>
References: <46E2D36D.4070509@ituniv.se>
Message-ID: <18195.25625.461057.577623@gargle.gargle.HOWL>

[Thomas Arts:]
> For cover analysis during testing it is important to see each clause
> in a case statement on a separate line.
<cut>
> I patched my erl_pp to contain:
> clauses(Type, Hook, Cs) ->
>     {prefer_nl,[$;],lexprs(Cs, Type, Hook)}.
> 
> from the original
> clauses(Type, Hook, Cs) ->
>     expr_list(Cs, [$;], Type, Hook). 
> 
> Would that be a good patch or are there arguments against it that I
> haven't considered?

Seems like a good idea. We will make the patch in R12B.

In order to pretty print

    t() -> receive after 1 -> ok end.

one more modification is needed:

f({prefer_nl,Sep,LItems}, I0, ST, WT) ->
    CharsSize2L = fl(LItems, Sep, I0, [], ST, WT),
    {_CharsL,Sizes} = unz(CharsSize2L),
    if
        Sizes =:= [] ->
            {[], 0};
        true ->
            {insert_newlines(CharsSize2L, I0, ST),nsz(lists:last(Sizes), I0)}
    end;

BTW, erl_pp will be able to handle binary comprehensions in R12B.

Thanks,

Hans Bolinder, Erlang/OTP team


From chris.newcombe@REDACTED  Mon Oct 15 17:55:55 2007
From: chris.newcombe@REDACTED (Chris Newcombe)
Date: Mon, 15 Oct 2007 08:55:55 -0700
Subject: [erlang-questions] Erlang BDB driver build issues
Message-ID: <781dd98c0710150855p667d2de2o5f11ce7f40064448@mail.gmail.com>

Hi Ciprian,

I am cc'ing the erlang-questions mailing list as it is possible that
my responses may help other people.


> 1) You are using a tool named "findup".
>    The problem is that this tool is not installed by default on my
> system -- Debian Etch.

This part of the makefile comes from the original EDTK makefiles,
which I didn't write.  I use RHEL3, and I found that the makefile just
worked.  I believe that the original author of the makefile used
Solaris.

I just checked, and 'findup' is actually a perl script in my company's
environment
Hopefully you can adapt this:

  #!/bin/perl5.8/perl
  use strict;
  use warnings;

  use Cwd;
  use File::Basename;

  my $target = shift || die "Usage: $0 <target-name>\n";
  my $dir = cwd;

  while ((! -e "$dir/$target") && ($dir ne dirname($dir))) {
    $dir = dirname($dir);
  }

  print $dir if -e "$dir/$target";


>    2) It seems that there are some hard-coded paths to pcre and
> coredumper libs. Also the make file assumes that the header files for
> both libraries are in a standard location. (Which in my case is not.)

Yes this is far from ideal and I apologise.
I hope that you saw that there is an installer script available that
includes patches that removes the absolute paths:

  http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-compilation-help


>    *) I have tried to compile it against BDB 4.6.21 -- the latest --
> but there were some compilation errors. After I tried with 4.5.20 --
> unpatched -- I seen that the error messages are the same. Maybe the
> patches wasn't accepted in the final release...

This won't work, and is actually dangerous.

You MUST use the build of BDB that is recommended in the
README-cnewcom file.   That is v4.5.20 plus the 20 or so patches
applied by the script in:

  examples/berkeley_db/patches-to-berkeley-db/db-4.5.20/apply-patches.sh

All of those patches (apart from one minor patch) are official Oracle
patches, to fix bugs in the released version of v4.5.20.   The patches
are essential for correct operation (particularly replication, and
also distributed transactions).   The same fixes have been included in
BDB v4.6, but Oracle do not normally back-port fixes to older existing
releases.  So the existing (final) v4.5.20 tarballs do not have these
patches. (i.e. Oracle did not do another release of v4.5 that includes
these patches -- you have to use the patch script above.)

There are constants in the examples/berkeley_db/berkeley_db.xml file
that are taken from the db.h file that is generated by the bdb build.
 To upgrade the driver to a new version of BDB, those constants must
be edited.  There are instructions in the berkeley_db.h file.  As BDB
releases are only once a year, I never automated this.

FYI There is an engineer at Oracle who has experimentally upgraded the
driver to use the latest BDB v4.6 release (they are evaluating Erlang
as an internal test platform for BDB-HA).  I can ask her for patches
if you like.   However, it would be a grave mistake to upgrade the
driver to BDB v4.6 until full testing has been done, as v4.6 is still
relatively new -- there have been a couple of patch releases already.
 So the patch would be marked 'experimental' at the very least.

I hope this helps,

Chris


On 10/12/07, Ciprian Dorin Craciun <ciprian.craciun@REDACTED> wrote:
>    Hello!
>
>    My name is Ciprian Craciun and I have read your posts on the
> Erlang discussion list.
>
>    I have tried to compile the driver but I have encountered some
> problems... Most of them I hacked out and managed to proceed... But
> still all these problems could make the difference if some one would
> like to asses your library.
>
>    I am writing this because maybe you could solve these
> inconveniences for further releases.
>
>    (I am using the latest version of EDTK -- 1.5.1)
>
>    1) You are using a tool named "findup".
>    The problem is that this tool is not installed by default on my
> system -- Debian Etch. Also the only binary named like this in the
> entire distribution does something completely different.
>    You could either remove it and use something else, or provide
> it... Even quick search on Google comes up with nothing useful...
>
>    2) It seems that there are some hard-coded paths to pcre and
> coredumper libs. Also the make file assumes that the header files for
> both libraries are in a standard location. (Which in my case is not.)
>    It would be nice to have these paths as variables, just like
> BDB_INSTALLED_DIR.
>    Maybe you could create a file named config that would be read by
> the make files and the shell scripts.
>
>    *) I have tried to compile it against BDB 4.6.21 -- the latest --
> but there were some compilation errors. After I tried with 4.5.20 --
> unpatched -- I seen that the error messages are the same. Maybe the
> patches wasn't accepted in the final release...
>
>    Except of these problems the build was successful and the first
> example worked (the one without replication). Now I have to play with
> the library.
>
>    I hope that this information is of some use to you.
>
>    Ciprian.


From per@REDACTED  Mon Oct 15 19:08:00 2007
From: per@REDACTED (Per Hedeland)
Date: Mon, 15 Oct 2007 19:08:00 +0200 (CEST)
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <47133AFE.1070700@lshift.net>
Message-ID: <200710151708.l9FH80Vl086969@pluto.hedeland.org>

Tony Garnock-Jones <tonyg@REDACTED> wrote:
>
>Per Hedeland wrote:
>> Hm, I don't think that's correct - AFAIK SIGPIPE/EPIPE happens only if
>> you try to write more data to a socket when the other end is considered
>> to be closed (either due to receiving FIN or RST, or due to
>> retransmission timeout).
>> 
>> Closing the socket with unack'ed data normally doesn't do anything in
>> particular, the close() returns immediately but the kernel holds on to
>> the data and tries to (re)transmit it according to the standard TCP
>> logic, and you won't be told if it fails.
>
>Thanks, Per. That's useful. It tells me that TCP itself can do what I
>want, but the sockets interface isn't set up for it.

Uh, you snipped the part where I described how the sockets interface
*is* set up for it:-) (via SO_LINGER).

> I'll do something different at the application level.

That's what you generally need to do anyway, if you want to be sure that
the remote *application* received that last chunk of data you sent - and
not just that the TCP stack ack'ed it.

--Per


From ciprian.craciun@REDACTED  Mon Oct 15 22:08:29 2007
From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun)
Date: Mon, 15 Oct 2007 23:08:29 +0300
Subject: [erlang-questions] Erlang BDB driver build issues
In-Reply-To: <781dd98c0710150855p667d2de2o5f11ce7f40064448@mail.gmail.com>
References: <781dd98c0710150855p667d2de2o5f11ce7f40064448@mail.gmail.com>
Message-ID: <8e04b5820710151308s42ca6468s783efb67593f1795@mail.gmail.com>

    Hello Chris!

    (See my answers below...)

On 10/15/07, Chris Newcombe <chris.newcombe@REDACTED> wrote:
> Hi Ciprian,
>
> I am cc'ing the erlang-questions mailing list as it is possible that
> my responses may help other people.
>
>
> > 1) You are using a tool named "findup".
> >    The problem is that this tool is not installed by default on my
> > system -- Debian Etch.
>
> This part of the makefile comes from the original EDTK makefiles,
> which I didn't write.  I use RHEL3, and I found that the makefile just
> worked.  I believe that the original author of the makefile used
> Solaris.
>
> I just checked, and 'findup' is actually a perl script in my company's
> environment
> Hopefully you can adapt this:
>
>   #!/bin/perl5.8/perl
>   use strict;
>   use warnings;
>
>   use Cwd;
>   use File::Basename;
>
>   my $target = shift || die "Usage: $0 <target-name>\n";
>   my $dir = cwd;
>
>   while ((! -e "$dir/$target") && ($dir ne dirname($dir))) {
>     $dir = dirname($dir);
>   }
>
>   print $dir if -e "$dir/$target";
>
>
> >    2) It seems that there are some hard-coded paths to pcre and
> > coredumper libs. Also the make file assumes that the header files for
> > both libraries are in a standard location. (Which in my case is not.)
>
> Yes this is far from ideal and I apologise.
> I hope that you saw that there is an installer script available that
> includes patches that removes the absolute paths:
>
>   http://www.snookles.com/erlang/edtk/edtk-1.5.1.README-compilation-help

    Yes, I have seen from previous posts that there is an installer
script. But for now I hadn't had time to try it. Finally I have
managed to install it.

> >    *) I have tried to compile it against BDB 4.6.21 -- the latest --
> > but there were some compilation errors. After I tried with 4.5.20 --
> > unpatched -- I seen that the error messages are the same. Maybe the
> > patches wasn't accepted in the final release...
>
> This won't work, and is actually dangerous.
>
> You MUST use the build of BDB that is recommended in the
> README-cnewcom file.   That is v4.5.20 plus the 20 or so patches
> applied by the script in:
>
>   examples/berkeley_db/patches-to-berkeley-db/db-4.5.20/apply-patches.sh
>
> All of those patches (apart from one minor patch) are official Oracle
> patches, to fix bugs in the released version of v4.5.20.   The patches
> are essential for correct operation (particularly replication, and
> also distributed transactions).   The same fixes have been included in
> BDB v4.6, but Oracle do not normally back-port fixes to older existing
> releases.  So the existing (final) v4.5.20 tarballs do not have these
> patches. (i.e. Oracle did not do another release of v4.5 that includes
> these patches -- you have to use the patch script above.)

    Indeed by applying the patches on v4.5.20 it compiled correctly.
But it seems that the patches are not included in v4.6.21, as the same
compilation error is encountered as with the unpatched v4.5.20... I
haven't tried to apply the patches on v4.6.21 -- maybe it would
work...

> There are constants in the examples/berkeley_db/berkeley_db.xml file
> that are taken from the db.h file that is generated by the bdb build.
>  To upgrade the driver to a new version of BDB, those constants must
> be edited.  There are instructions in the berkeley_db.h file.  As BDB
> releases are only once a year, I never automated this.

    It's Ok, and also I think the constants do not change from one
release to another...

> FYI There is an engineer at Oracle who has experimentally upgraded the
> driver to use the latest BDB v4.6 release (they are evaluating Erlang
> as an internal test platform for BDB-HA).  I can ask her for patches
> if you like.   However, it would be a grave mistake to upgrade the
> driver to BDB v4.6 until full testing has been done, as v4.6 is still
> relatively new -- there have been a couple of patch releases already.
>  So the patch would be marked 'experimental' at the very least.

    I would very much like to obtain the latest patches for v4.6...
Also I am not currently planing to use BerkeleyDB with Erlang in a
production application... I just want to play with them -- mostly
Erlang... (I am new to Erlang...)

    But back to the topic, I am available to test the new version...

> I hope this helps,
>
> Chris
>
>
> On 10/12/07, Ciprian Dorin Craciun <ciprian.craciun@REDACTED> wrote:
> >    Hello!
> >
> >    My name is Ciprian Craciun and I have read your posts on the
> > Erlang discussion list.
> >
> >    I have tried to compile the driver but I have encountered some
> > problems... Most of them I hacked out and managed to proceed... But
> > still all these problems could make the difference if some one would
> > like to asses your library.
> >
> >    I am writing this because maybe you could solve these
> > inconveniences for further releases.
> >
> >    (I am using the latest version of EDTK -- 1.5.1)
> >
> >    1) You are using a tool named "findup".
> >    The problem is that this tool is not installed by default on my
> > system -- Debian Etch. Also the only binary named like this in the
> > entire distribution does something completely different.
> >    You could either remove it and use something else, or provide
> > it... Even quick search on Google comes up with nothing useful...
> >
> >    2) It seems that there are some hard-coded paths to pcre and
> > coredumper libs. Also the make file assumes that the header files for
> > both libraries are in a standard location. (Which in my case is not.)
> >    It would be nice to have these paths as variables, just like
> > BDB_INSTALLED_DIR.
> >    Maybe you could create a file named config that would be read by
> > the make files and the shell scripts.
> >
> >    *) I have tried to compile it against BDB 4.6.21 -- the latest --
> > but there were some compilation errors. After I tried with 4.5.20 --
> > unpatched -- I seen that the error messages are the same. Maybe the
> > patches wasn't accepted in the final release...
> >
> >    Except of these problems the build was successful and the first
> > example worked (the one without replication). Now I have to play with
> > the library.
> >
> >    I hope that this information is of some use to you.
> >
> >    Ciprian.

    Thank you for your time,
    Ciprian.


From kosik@REDACTED  Mon Oct 15 22:54:56 2007
From: kosik@REDACTED (Matej Kosik)
Date: Mon, 15 Oct 2007 22:54:56 +0200
Subject: [erlang-questions] report: two oddities
Message-ID: <4713D3A0.4020805@fiit.stuba.sk>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Friends (Erlangers),

(1)

I believe that very few people use `dwm'
http://www.suckless.org/wiki/dwm
"dynamic window manager"

but some Erlang GUI tools suffer in this environment.

- - When I am in "tiled layout"
- - and when I start "table viewer"

	tv:start()

- - and when the "table viewer window" is moved from "master area" into "stacking area" (on the right)

then the window periodically blinks because it tries to redraw itself in infinite loop.
- ------------------------------------------------------------
(2)

Apart from the fact how much debugger is useful (or not) there is a small oddity. If you open a
dialog for (menu: Modules / Interpret ...) this dialog box cannot be maximized---or better to say,
it always shows at most 5 files/directories at once. People have large screens and might perhaps
welcome the possibility to view more files/directories at once and browse them.

Regards
- --
Matej Ko??k
ICQ: 300133844
skype: matej_kosik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHE9OgL+CaXfJI/hgRAgfMAJ9P4kYfodeRvctlmlu3P/+EzzBoTwCeIeoX
7Fu5MrFhtpx6YJxgXBVuh7M=
=qbOE
-----END PGP SIGNATURE-----


From tuncer.ayaz@REDACTED  Mon Oct 15 23:07:31 2007
From: tuncer.ayaz@REDACTED (Tuncer Ayaz)
Date: Mon, 15 Oct 2007 23:07:31 +0200
Subject: [erlang-questions] report: two oddities
In-Reply-To: <4713D3A0.4020805@fiit.stuba.sk>
References: <4713D3A0.4020805@fiit.stuba.sk>
Message-ID: <4ac8254d0710151407s6afef4b2kfe12b1af289bbd83@mail.gmail.com>

On 10/15/07, Matej Kosik <kosik@REDACTED> wrote:

<snip>

> I believe that very few people use `dwm'
> http://www.suckless.org/wiki/dwm
> "dynamic window manager"

You can count me in as a dwm user being interested in
and playing around with Erlang :)

<snip>

You should also post about your problem on the dwm mailing list.

Although I don't use it you could try http://awesome.naquadah.org/
which was based on dwm IIRC but does not limit itself in SLOC and
simplicity.


From hubaghdadi@REDACTED  Tue Oct 16 00:43:28 2007
From: hubaghdadi@REDACTED (Lone Wolf)
Date: Mon, 15 Oct 2007 15:43:28 -0700 (PDT)
Subject: [erlang-questions] Erlang or OCaml
Message-ID: <536569.99564.qm@web51106.mail.re2.yahoo.com>

Hi.
Recently, I have been trying to learn a language other than Java.
Erlang seems to be ready for the prime time (used in the commercial world).
On the other side, OCaml seems to be a cool and capable language.
I want to learn something useful and fun.
Tell what do you think ?
(if you are Erlang die-hard, please put this aside ;))

    Deep into that darkness peering, long I stood there, wondering, fearing, Doubting, dreaming dreams no mortal ever dreamed before.
  E.A Poe
  
   



       
---------------------------------
Catch up on fall's hot new shows on Yahoo! TV.  Watch previews, get listings, and more!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071015/ff7820bf/attachment.htm>

From cyberlync@REDACTED  Tue Oct 16 01:42:54 2007
From: cyberlync@REDACTED (Eric Merritt)
Date: Mon, 15 Oct 2007 16:42:54 -0700
Subject: [erlang-questions] Erlounge Seattle Monday November 5th @ 6:30
Message-ID: <f69c840c0710151642l6c7674efkda6b2ae1cb5bee35@mail.gmail.com>

A few of us are getting together to drink beer, have a bite and discuss
Erlang (In that order) on Monday the 5th of November in Seattle, WA.
The venue is Barca, 1510 11th Avenue on Capitol Hill. More info on their
site http://www.barcaseattle.com/


From dustin@REDACTED  Tue Oct 16 02:18:05 2007
From: dustin@REDACTED (Dustin Sallings)
Date: Mon, 15 Oct 2007 17:18:05 -0700
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <8B7200FB-1499-49CC-841B-9FAF4AAD2BE8@spy.net>


On Oct 15, 2007, at 15:43 , Lone Wolf wrote:

> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial  
> world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))

	Why choose?  I've learned a lot from each language.

	I do admit that I've shipped more ocaml code than erlang code,  
though.  ocaml is great for building utilities and erlang is great  
for building systems.  It really depends on your task.

	OTP really seems to me like what java wants to be when it grows up.

-- 
Dustin Sallings


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071015/d0bdfb15/attachment.htm>

From michael.campbell@REDACTED  Tue Oct 16 02:18:17 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Mon, 15 Oct 2007 20:18:17 -0400
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <811f2f1c0710151718n1b9f3c00i60961b53bf93fbf1@mail.gmail.com>

I'm going through the same search myself.  I haven't decided, so until
I do, I'm playing with both.  And Haskell.  With some Scala thrown in.
 =)

On 10/15/07, Lone Wolf <hubaghdadi@REDACTED> wrote:
> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
>
>
>
> Deep into that darkness peering, long I stood there, wondering, fearing,
> Doubting, dreaming dreams no mortal ever dreamed before.
> E.A Poe
>
>
>
>  ________________________________
> Catch up on fall's hot new shows on Yahoo! TV. Watch previews, get listings,
> and more!
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From berlin.brown@REDACTED  Tue Oct 16 02:34:46 2007
From: berlin.brown@REDACTED (Berlin Brown)
Date: Mon, 15 Oct 2007 20:34:46 -0400
Subject: [erlang-questions] Basic questions, also related to yaws and mnesia
Message-ID: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>

I should probably move away from yaws for a sec and go over the
basics, but I am really close to what I want to do.

I have a record and a list of records, but I don't know the best ways
to extract the data and show the data out of the record.

For example, with this code; how would I iterate(map?) those links and
display them.

out(Args) ->
	{ok, EntityLinks} = botlist_find_links:find_links(),
	{ html,
	 	yaws_api:f("~p", [EntityLinks])
	}.

This is the record.
-record(entity_links, {main_url, url_title, keywords, rating,
full_name, created_on}).

-- 
Berlin Brown
http://botspiritcompany.com/botlist/spring/help/about.html
newspirit technologies


From yinso.chen@REDACTED  Tue Oct 16 03:24:43 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 15 Oct 2007 18:24:43 -0700
Subject: [erlang-questions] parallel tcp server closed once spawned
Message-ID: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>

Hi -

I'm going through the exercises in Programming Erlang, and I run into a
puzzling error with the parallel_server example.  Below is my almost
verbatim copy from the book.

para_server() ->
    {ok, Listen} = gen_tcp:listen(2345, [binary, {packet, 4},
                     {reuseaddr, true},
                     {active, true}]),
    spawn(fun () ->
          par_connect(Listen) end).

par_connect(Listen) ->
    {ok, Socket} = gen_tcp:accept(Listen),
    spawn(fun () ->
          par_connect(Listen) end),
    loop(Socket).

When I try to run the code, I ended up with a badmatch error.

(emacs@REDACTED)12> P5 = spawn(nano_server, para_server, []).
<0.67.0>

=ERROR REPORT==== 15-Oct-2007::18:04:30 ===
Error in process <0.68.0> on node 'emacs@REDACTED' with exit value:
{{badmatch,{error,closed}},[{nano_server,par_connect,1}]}

So gen_tcp:accept(Listen) returns {error, closed} instead of {ok, Socket},
but I'm not sure why, as the gen_tcp:listen line appears to run correctly
(and I've also verified that the seq_server version works).  I must be
missing something extremely obvious here.

Any thoughts are appreciated, thanks.
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071015/d3fcef04/attachment.htm>

From bob@REDACTED  Tue Oct 16 03:38:22 2007
From: bob@REDACTED (Bob Ippolito)
Date: Mon, 15 Oct 2007 18:38:22 -0700
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <6a36e7290710151838p1b99ad7blec3491e16c0016a5@mail.gmail.com>

On 10/15/07, Lone Wolf <hubaghdadi@REDACTED> wrote:
> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
>

When I was evaluating Python alternatives for building the core
technology behind MochiAds I tried out a bunch of languages and Erlang
was the only one that was easy for me to learn and had the right
balance of features, performance, and reliability. A year later we
have about 16 machines running 80 Erlang nodes powering about 16
different "components" of our infrastructure and 4 people working on
it at the moment (originally it was just me). It worked out so well
that we rewrote the server component of our MochiBot service in Erlang
and we've been using it to build lots of internal tools such as our
monitoring software, our single sign-on service, etc. as well. None of
us had previous Erlang experience, but we're all very comfortable with
it now.

After about a year with Erlang, I'm not sure I could part with hot
code loading, light-weight processes, and multiplexed socket IO for
writing servers. Also, Mnesia has been really useful to us to
temporarily store "real-time" data (ram_copies) so that we don't have
to make users wait for it to get batched into the SQL databases. The
distribution stuff mostly Just Works once you figure out how to set it
up (though we did have one bad experience with a network partition due
to a switch acting up, it was recoverable manually).

O'Caml is a useful language too, but for writing a network app I can't
really imagine going with anything but Erlang if you're looking for
redundancy and scale. Unless you want to write your own half-baked
Erlang-like system before even trying to solve something a little
closer to your actual problem domain.

-bob


From yinso.chen@REDACTED  Tue Oct 16 04:30:50 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 15 Oct 2007 19:30:50 -0700
Subject: [erlang-questions] parallel tcp server closed once spawned
In-Reply-To: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>
References: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>
Message-ID: <779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>

Update -

it turns out that if I call para_server directly instead of using spawn, it
works.  Why this is the case, I'm not fully certain, but my guess is that if
I call para_server via spawn, then the first process dies right after it
spawns the accept processes, and causes the connection to close, and hence
when par_connect is evaluated it finds the listener closed.

If I miss something, please let me know.  Thanks,
yc


> I'm going through the exercises in Programming Erlang, and I run into a
> puzzling error with the parallel_server example.  Below is my > almost
> verbatim copy from the book.
>
> When I try to run the code, I ended up with a badmatch error.
>
> (emacs@REDACTED)12> P5 = spawn(nano_server, para_server, []).
> <0.67.0>
>
> =ERROR REPORT==== 15-Oct-2007::18:04:30 ===
> Error in process <0.68.0> on node ' emacs@REDACTED' with exit value:
> {{badmatch,{error,closed}},[{nano_server,par_connect,1}]}
>
> > So gen_tcp:accept(Listen) returns {error, closed} instead of {ok,
> Socket}, but I'm not sure why, as the  > gen_tcp:listen line appears to
> run correctly (and I've also verified that the seq_server version works).  I
> must be missing something > extremely obvious here.
>
> > Any thoughts are appreciated, thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071015/d7c9df20/attachment.htm>

From yinso.chen@REDACTED  Tue Oct 16 06:57:47 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 15 Oct 2007 21:57:47 -0700
Subject: [erlang-questions] application:is_started?
Message-ID: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>

Hi all -
I've noticed that the application library doesn't provide an equivalent of a
is_started function, so I'm wondering whether there is an idiom to check
whether an app is running:

1) Are functions expected to derived the result from checking {ok, Pid} or
{error, already_started} everytime they want to use an application? or

2) Are functions expected to simply call the interface provided by the app
w/out regard to whether the app is started? or

3) Are there ways to ensure the application in question is started.

I'm thinking of library modules where each function can be called
independently but depend on some underlying applications.  An example is a
wrapper onto of the crypto app - because crypto needs to be started, either
I have to make the wrapper into another server or I would have to test
whether the crypto app is started on every function.

Would love to see what type of code structure is "good" - any thoughts are
appreciated.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071015/ea284e94/attachment.htm>

From chandrashekhar.mullaparthi@REDACTED  Tue Oct 16 11:36:27 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Tue, 16 Oct 2007 10:36:27 +0100
Subject: [erlang-questions] Basic questions,
	also related to yaws and mnesia
In-Reply-To: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
References: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
Message-ID: <b447a30d0710160236l21e9cf9bkc311f24af760e72c@mail.gmail.com>

On 16/10/2007, Berlin Brown <berlin.brown@REDACTED> wrote:
> I should probably move away from yaws for a sec and go over the
> basics, but I am really close to what I want to do.
>
> I have a record and a list of records, but I don't know the best ways
> to extract the data and show the data out of the record.
>
> For example, with this code; how would I iterate(map?) those links and
> display them.
>
> out(Args) ->
>         {ok, EntityLinks} = botlist_find_links:find_links(),
>         { html,
>                 yaws_api:f("~p", [EntityLinks])
>         }.
>
> This is the record.
> -record(entity_links, {main_url, url_title, keywords, rating,
> full_name, created_on}).
>

out(Args) ->
        {ok, EntityLinks} = botlist_find_links:find_links(),
        { html,
                [make_link(X) || X <- EntityLinks]
        }.

make_link(#entity_link{main_url = Url, url_title = Title}) ->
    io_lib:format("<a href=~s>~s</a>", [Url, Title]).

Or if you want to squeeze out every ounce of performance:
make_link(#entity_link{main_url = Url, url_title = Title}) ->
    ["a href=", Url, ">", Title, "</a>"].

cheers
Chandru


From chandrashekhar.mullaparthi@REDACTED  Tue Oct 16 11:46:10 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Tue, 16 Oct 2007 10:46:10 +0100
Subject: [erlang-questions] parallel tcp server closed once spawned
In-Reply-To: <779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>
References: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>
	<779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>
Message-ID: <b447a30d0710160246u77f3f5d9rca0cb44971f8f6b2@mail.gmail.com>

On 16/10/2007, YC <yinso.chen@REDACTED> wrote:
> Update -
>
> it turns out that if I call para_server directly instead of using spawn, it
> works.  Why this is the case, I'm not fully certain, but my guess is that if
> I call para_server via spawn, then the first process dies right after it
> spawns the accept processes, and causes the connection to close, and hence
> when par_connect is evaluated it finds the listener closed.
>
> If I miss something, please let me know.  Thanks,
> yc
>

You are right, when you spawn, the process which created the socket
dies and the socket is closed. But when you directly call para_server
from the shell, your shell process owns the socket. And as long as you
don't make any mistakes in the shell when typing, your socket will
remain open! See below. I've included a modified version of the
para_server which will work whether you spawn or call it directly from
the shell.

2> par_server:para_server().
<0.45.0>
3>
3> inet:i().
Port Module    Recv Sent Owner    Local Address Foreign Address State
129  inet_tcp  0    0    <0.38.0> *:dbm         *:*             LISTENING
130  undefined 0    0    <0.45.0>               *:*             ACCEPTING
Port Module Recv Sent Owner Local Address Foreign Address State
ok
4> 1/0.

=ERROR REPORT==== 16-Oct-2007::10:41:32 ===
Error in process <0.38.0> with exit value:
{badarith,[{erl_eval,eval_op,3},{shell,exprs,6},{shell,eval_loop,3}]}

** exited: {badarith,[{erl_eval,eval_op,3},
                      {shell,exprs,6},
                      {shell,eval_loop,3}]} **

=ERROR REPORT==== 16-Oct-2007::10:41:32 ===
Error in process <0.45.0> with exit value:
{{badmatch,{error,closed}},[{par_server,par_connect,1}]}

>
>
> > > I'm going through the exercises in Programming Erlang, and I run into a
> puzzling error with the parallel_server example.  Below is my > almost
> verbatim copy from the book.
> >
> > > When I try to run the code, I ended up with a badmatch error.
> >
> >
> > (emacs@REDACTED)12> P5 = spawn(nano_server, para_server, []).
> > <0.67.0>
> >
> > =ERROR REPORT==== 15-Oct-2007::18:04:30 ===
> > Error in process <0.68.0> on node ' emacs@REDACTED' with exit value:
> {{badmatch,{error,closed}},[{nano_server,par_connect,1}]}
> >
> > > So gen_tcp:accept(Listen) returns {error, closed} instead of {ok,
> Socket}, but I'm not sure why, as the  > gen_tcp:listen line appears to run
> correctly (and I've also verified that the seq_server version works).  I
> must be missing something > extremely obvious here.
> >
> > > Any thoughts are appreciated, thanks.

para_server() ->
    spawn(fun() ->
		  para_server_1()
	  end).

para_server_1() ->
    {ok, Listen} = gen_tcp:listen(2345, [binary, {packet, 4},
					 {reuseaddr, true},
					 {active, true}]),
    par_connect(Listen).

par_connect(Listen) ->
    {ok, Socket} = gen_tcp:accept(Listen),
    spawn(fun () ->
          par_connect(Listen) end),
    loop(Socket).

loop(Socket) ->
    ok.

cheers
Chandru


From chandrashekhar.mullaparthi@REDACTED  Tue Oct 16 11:47:28 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Tue, 16 Oct 2007 10:47:28 +0100
Subject: [erlang-questions] application:is_started?
In-Reply-To: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>
References: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>
Message-ID: <b447a30d0710160247l133883dbx49723bcb073617a7@mail.gmail.com>

On 16/10/2007, YC <yinso.chen@REDACTED> wrote:
> Hi all -
> I've noticed that the application library doesn't provide an equivalent of a
> is_started function, so I'm wondering whether there is an idiom to check
> whether an app is running:
>
> 1) Are functions expected to derived the result from checking {ok, Pid} or
> {error, already_started} everytime they want to use an application? or
>
> 2) Are functions expected to simply call the interface provided by the app
> w/out regard to whether the app is started? or
>
> 3) Are there ways to ensure the application in question is started.
>

6> application:info().
[{loaded,[{kernel,"ERTS  CXC 138 10","2.10.6"},
          {stdlib,"ERTS  CXC 138 10","1.13.5"},
          {sasl,"SASL  CXC 138 11","2.0.1"}]},
 {loading,[]},
 {started,[{sasl,permanent},{stdlib,permanent},{kernel,permanent}]},
 {start_p_false,[]},
 {running,[{sasl,<0.28.0>},{stdlib,undefined},{kernel,<0.7.0>}]},
 {starting,[]}]


From saleyn@REDACTED  Tue Oct 16 14:24:02 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Tue, 16 Oct 2007 07:24:02 -0500
Subject: [erlang-questions] application:is_started?
In-Reply-To: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>
References: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>
Message-ID: <4714AD62.7090108@gmail.com>

YC wrote:
> Hi all -
> I've noticed that the application library doesn't provide an equivalent of a
> is_started function, so I'm wondering whether there is an idiom to check
> whether an app is running:
> 
> 1) Are functions expected to derived the result from checking {ok, Pid} or
> {error, already_started} everytime they want to use an application? or

1> App = stdlib.
stdlib
2> [T || {A, _, _} = T <- application:which_applications(), A =:= App].
[{stdlib,"ERTS  CXC 138 10","1.14.4"}]

If you need to expect if an application is running from within the
modules of that application, use application:get_application/0.

> 2) Are functions expected to simply call the interface provided by the app
> w/out regard to whether the app is started? or

I would say yes - it is the job of a proper boot script and following
design principles on application and supervisor behaviors to ensure that
your run-time environment is started as needed.

> 3) Are there ways to ensure the application in question is started.

Use a boot script (produced from a *.rel file using erlc), and start the
emulator with a "-boot your_boot_script" option.

> I'm thinking of library modules where each function can be called
> independently but depend on some underlying applications.  An example is a
> wrapper onto of the crypto app - because crypto needs to be started, either
> I have to make the wrapper into another server or I would have to test
> whether the crypto app is started on every function.
> 
> Would love to see what type of code structure is "good" - any thoughts are
> appreciated.
> 
> Thanks,
> yc
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions




From tonyg@REDACTED  Tue Oct 16 14:26:17 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Tue, 16 Oct 2007 13:26:17 +0100
Subject: [erlang-questions] EPIPE and TCP sockets
In-Reply-To: <200710151708.l9FH80Vl086969@pluto.hedeland.org>
References: <200710151708.l9FH80Vl086969@pluto.hedeland.org>
Message-ID: <4714ADE9.9020900@lshift.net>

Per Hedeland wrote:
> Uh, you snipped the part where I described how the sockets interface
> *is* set up for it:-) (via SO_LINGER).

Well... ok :-) Yeah. It's just that I'd be uncomfortable using it in
erlang, as it stands, and also...

> That's what you generally need to do anyway, if you want to be sure that
> the remote *application* received that last chunk of data you sent - and
> not just that the TCP stack ack'ed it.

... this rather puts the fatal spanner in the works, as you observe.

I'll definitely be sticking with the app-level solution :-)

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg@REDACTED


From michael.campbell@REDACTED  Tue Oct 16 14:40:47 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Tue, 16 Oct 2007 08:40:47 -0400
Subject: [erlang-questions] parallel tcp server closed once spawned
In-Reply-To: <b447a30d0710160246u77f3f5d9rca0cb44971f8f6b2@mail.gmail.com>
References: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>
	<779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>
	<b447a30d0710160246u77f3f5d9rca0cb44971f8f6b2@mail.gmail.com>
Message-ID: <811f2f1c0710160540off5fcfbs232e814ee48a9b78@mail.gmail.com>

As a tangent to this conversation...

On 10/16/07, Chandru <chandrashekhar.mullaparthi@REDACTED> wrote:
> I've included a modified version of the
> para_server which will work whether you spawn or call it directly from
> the shell.


> para_server() ->
>     spawn(fun() ->
>                   para_server_1()
>           end).


Is there a difference in the spawn call above and:

spawn(fun() ->
            para_server_1/0
end).

?   I've seen in other examples the M:F/A format for methods that take
a fun, and wondered if this were a stylistic or subtle "real"
difference.


From bbmaj7@REDACTED  Tue Oct 16 13:44:39 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Tue, 16 Oct 2007 21:44:39 +1000 (EST)
Subject: [erlang-questions] systools:make_script performs a DNS lookup
Message-ID: <177488.15389.qm@web52003.mail.re2.yahoo.com>

While tracing unexplained pauses on my development system, I found that
systools:make_script is doing a DNS lookup for host 'ebin.mydomain' where
mydomain is the domain of the development host.

Why would make_script use DNS and how can I prevent it from doing so?

Using R11B5 under x86 Linux (fc4 and fc6)



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From bengt.kleberg@REDACTED  Tue Oct 16 14:50:18 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Tue, 16 Oct 2007 14:50:18 +0200
Subject: [erlang-questions] parallel tcp server closed once spawned
In-Reply-To: <811f2f1c0710160540off5fcfbs232e814ee48a9b78@mail.gmail.com>
References: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>	<779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>	<b447a30d0710160246u77f3f5d9rca0cb44971f8f6b2@mail.gmail.com>
	<811f2f1c0710160540off5fcfbs232e814ee48a9b78@mail.gmail.com>
Message-ID: <4714B38A.2000901@ericsson.com>

if you write para_server_1/0 you are dividing an atom (para_server_1) 
with 0. that is a sure way to crash.
luckily enough you are doing it in a seperate process and the rest of 
your applicaiton will survive :-)


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-16 14:40, Michael Campbell wrote:
> As a tangent to this conversation...
> 
> On 10/16/07, Chandru <chandrashekhar.mullaparthi@REDACTED> wrote:
>> I've included a modified version of the
>> para_server which will work whether you spawn or call it directly from
>> the shell.
> 
> 
>> para_server() ->
>>     spawn(fun() ->
>>                   para_server_1()
>>           end).
> 
> 
> Is there a difference in the spawn call above and:
> 
> spawn(fun() ->
>             para_server_1/0
> end).
> 
> ?   I've seen in other examples the M:F/A format for methods that take
> a fun, and wondered if this were a stylistic or subtle "real"
> difference.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From ulf.wiger@REDACTED  Tue Oct 16 14:50:48 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Tue, 16 Oct 2007 14:50:48 +0200
Subject: [erlang-questions] parallel tcp server closed once spawned
In-Reply-To: <811f2f1c0710160540off5fcfbs232e814ee48a9b78@mail.gmail.com>
References: <779bf2730710151824h4d106870x3d44a360171eb686@mail.gmail.com>	<779bf2730710151930i7ad83f83h31a9fb4b443693a1@mail.gmail.com>	<b447a30d0710160246u77f3f5d9rca0cb44971f8f6b2@mail.gmail.com>
	<811f2f1c0710160540off5fcfbs232e814ee48a9b78@mail.gmail.com>
Message-ID: <4714B3A8.6070309@ericsson.com>

Michael Campbell wrote:
>> para_server() ->
>>     spawn(fun() ->
>>                   para_server_1()
>>           end).
> 
> 
> Is there a difference in the spawn call above and:
> 
> spawn(fun() ->
>             para_server_1/0
> end).

That would be spawn(fun para_server_1/0).

The way you wrote it, you'd be trying to divide
an atom by zero.

To my knowledge, there isn't any difference.

BR,
Ulf W


From pacini@REDACTED  Tue Oct 16 15:23:35 2007
From: pacini@REDACTED (Filippo Pacini)
Date: Tue, 16 Oct 2007 15:23:35 +0200
Subject: [erlang-questions] Basic questions,
 also related to yaws and mnesia
In-Reply-To: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
References: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
Message-ID: <4714BB57.9080206@sgconsulting.it>

Hi,
I'm a little partisan here :-), but you might be interested in sgte
http://sgte.googlecode.com

It' s a simple template language. In your example you could write
something like this:

gen_html(Entities) ->
Data = [{entities, [sgte:rec_to_kv(Rec, record_info(fields,
entity_links)) || Rec <- Entities]}],
{ok, C} = sgte:compile("$map:{<a href=\"$main_url$\">$url_title$</a>}
entities$"),
sgte:render(C, Data)}.

The template instead of a string can also be an html file, so you can
write html templates containing sgte instructions and you pass the data
to be rendered at runtime.

cheers,
filippo


Berlin Brown wrote:
> I should probably move away from yaws for a sec and go over the
> basics, but I am really close to what I want to do.
> 
> I have a record and a list of records, but I don't know the best ways
> to extract the data and show the data out of the record.
> 
> For example, with this code; how would I iterate(map?) those links and
> display them.
> 
> out(Args) ->
> 	{ok, EntityLinks} = botlist_find_links:find_links(),
> 	{ html,
> 	 	yaws_api:f("~p", [EntityLinks])
> 	}.
> 
> This is the record.
> -record(entity_links, {main_url, url_title, keywords, rating,
> full_name, created_on}).
> 



From james.hague@REDACTED  Tue Oct 16 15:50:48 2007
From: james.hague@REDACTED (James Hague)
Date: Tue, 16 Oct 2007 08:50:48 -0500
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <e7cc45650710160650h3f0ac474k738a28cf2d6d80bc@mail.gmail.com>

The short version is that they're both good languages.

If raw, single-processor speed was key for me, then I'd use OCaml.
Ditto for problems that make heavy use of array updates (like image
processing).  I also program in C and Forth, so I might use one of
those, depending on the situation.

That said, I typically use Erlang as my personal project
language--this *is* an Erlang mailing list, after all--for two
reasons.  First, the core language agrees with me more: dynamic
typing, binaries, overall syntax.  Second, Erlang is a better fit for
the modern world: lightweight processes, ability to reload code while
an application is running (wonderful for development!).

I spent quite a bit of time back in 1998-9 or so evaluating Haskell,
Erlang, OCaml, SML/NJ, Lisp, etc., and have stuck with Erlang since
then.


From kenneth.lundin@REDACTED  Tue Oct 16 19:47:59 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Tue, 16 Oct 2007 19:47:59 +0200
Subject: [erlang-questions] systools:make_script performs a DNS lookup
In-Reply-To: <177488.15389.qm@web52003.mail.re2.yahoo.com>
References: <177488.15389.qm@web52003.mail.re2.yahoo.com>
Message-ID: <de422a40710161047x4ead0e6ci20290c50a9a22b4a@mail.gmail.com>

Hi,

And how are you running systools:make_script when you observe the DNS lookup?
Are you perhaps running it from the command line something like this?

erl -s systools makescript SomeMoreArg

I agree that there is no reason for systools:make_script to do a DNS lookup.
It must be a side effect of something else.

/Kenneth (Erlang/OTP group at Ericsson)

On 10/16/07, Richard Andrews <bbmaj7@REDACTED> wrote:
> While tracing unexplained pauses on my development system, I found that
> systools:make_script is doing a DNS lookup for host 'ebin.mydomain' where
> mydomain is the domain of the development host.
>
> Why would make_script use DNS and how can I prevent it from doing so?
>
> Using R11B5 under x86 Linux (fc4 and fc6)
>
>
>
>      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
> http://au.docs.yahoo.com/mail/unlimitedstorage.html
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From sten@REDACTED  Tue Oct 16 20:30:13 2007
From: sten@REDACTED (Sten Kvamme)
Date: Tue, 16 Oct 2007 20:30:13 +0200
Subject: [erlang-questions] double to float
Message-ID: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>

Hello,
I'm new at this list but have been lurking for a while. I bought the  
new Erlang book and are starting to learn the language, and I have a  
question:

I'm reading a binary file with lots of double data types. Have  
someone already written some code to convert the double 8 bytes,  
<<184,162,71,68,75,65,104,64>> to a float?

Thanks
Sten


From vladdu55@REDACTED  Tue Oct 16 20:43:06 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 16 Oct 2007 18:43:06 +0000
Subject: [erlang-questions] double to float
In-Reply-To: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
Message-ID: <95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>

Hi and welcome,

On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
> I'm reading a binary file with lots of double data types. Have
> someone already written some code to convert the double 8 bytes,
> <<184,162,71,68,75,65,104,64>> to a float?

Do you mean
   <<X/float>> = <<184,162,71,68,75,65,104,64>>
?

regards,
Vlad


From sten@REDACTED  Tue Oct 16 22:32:09 2007
From: sten@REDACTED (Sten Kvamme)
Date: Tue, 16 Oct 2007 22:32:09 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
Message-ID: <6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>

Yes thanks, but I have the file in a binary variable and would like  
to do something like this:

{<<F/float>>,B} = split_binary(Thefile, 8),

or maybe

<<F:8/binary,B/binary>> = Thefile,
<<X/float>> =  F,

/Sten

On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:

> <<X/float>> = <<184,162,71,68,75,65,104,64>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071016/597818fa/attachment.htm>

From mogorman@REDACTED  Tue Oct 16 22:41:42 2007
From: mogorman@REDACTED (Matthew O'Gorman)
Date: Tue, 16 Oct 2007 15:41:42 -0500
Subject: [erlang-questions] double to float
In-Reply-To: <6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>
Message-ID: <f49fd40d0710161341s361d682fw74d5496ca5e19850@mail.gmail.com>

this works great if you compile with binary_comprehension

List_of_floats = [Float || << Float/float >> <= Binary].

mog
On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
> Yes thanks, but I have the file in a binary variable and would like to do
> something like this:
>
> {<<F/float>>,B} = split_binary(Thefile, 8),
>
> or maybe
>
> <<F:8/binary,B/binary>> = Thefile,
> <<X/float>> =  F,
>
>
> /Sten
>
>
> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>
>
> <<X/float>> = <<184,162,71,68,75,65,104,64>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From yinso.chen@REDACTED  Wed Oct 17 00:45:59 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 16 Oct 2007 15:45:59 -0700
Subject: [erlang-questions] application:is_started?
In-Reply-To: <4714AD54.2080305@gmail.com>
References: <779bf2730710152157y5e3c597ao6ad38cce73bc3254@mail.gmail.com>
	<4714AD54.2080305@gmail.com>
Message-ID: <779bf2730710161545p5b02599yb26fd0f35a1f0c63@mail.gmail.com>

On 10/16/07, Serge Aleynikov <saleyn@REDACTED> wrote:
>
> > > 3) Are there ways to ensure the application in question is started.
>
> > Use a boot script (produced from a *.rel file using erlc), and start the
> > emulator with a "-boot your_boot_script" option.


Thanks - I missed the concept of boot script and rel file, but this sounds
like the correct segmentation of responsibilities.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071016/bd7aa95d/attachment.htm>

From rpettit@REDACTED  Wed Oct 17 06:05:36 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Tue, 16 Oct 2007 23:05:36 -0500
Subject: [erlang-questions] why might mnesia:start() hang?
Message-ID: <20071017040536.GB19639@vailsys.com>

I seem to have encountered a situation in which I am unable to start mnesia.
Attempts to start mnesia (via mnesia:start/0) hang the erlang shell.

In the scenario below there are 2 physical servers, each running an instance
of the foo_rel and bar_rel. The second physical server, someother.somedomain,
has been halted prior to starting the nodes somebox.somedomain.

The foo_rel instances contain disc_copy tables--bar_rel instances contain
ram_copies only.

(foo_rel@REDACTED)1> application:which_applications().
[{sasl,"SASL  CXC 138 11","2.1.5.1"},
 {stdlib,"ERTS  CXC 138 10","1.14.5"},
 {kernel,"ERTS  CXC 138 10","2.11.5"}]


   NOTE: there are other applications in this release which *should* be running
         but are not, almost certainly due to the fact that mnesia is refusing
         to start


(foo_rel@REDACTED)2> mnesia:info().
===> System info in version "4.3.5", debug level = none <===
opt_disc. Directory "/u1/otp/db/foo_rel" is used.
use fallback at restart = false
running db nodes   = ['foo_rel@REDACTED']
stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
ok


(foo_rel@REDACTED)3> mnesia:stop().
stopped


(foo_rel@REDACTED)4> mnesia:start().
...shell hangs forever...


Shell back into the node, try again:


(foo_rel@REDACTED)1> mnesia:info().
===> System info in version "4.3.5", debug level = none <===
opt_disc. Directory "/u1/otp/db/foo_rel" is used.
use fallback at restart = false
running db nodes   = ['foo_rel@REDACTED']
stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
ok


(foo_rel@REDACTED)2> application:which_applications().
[{sasl,"SASL  CXC 138 11","2.1.5.1"},
 {stdlib,"ERTS  CXC 138 10","1.14.5"},
 {kernel,"ERTS  CXC 138 10","2.11.5"}]


(foo_rel@REDACTED)3> mnesia:start().
...hangs forever...


======

I cannot seem to figure out:

  1) why mnesia refuses to start

  2) why mnesia:start() hangs forever at the shell (vs. return an error, etc)

Any applications requiring mnesia tables do a mnesia:wait_for_tables/2 on them.

A special process performs a mnesia:force_load_table/1, if necessary (e.g. when
wait_for_tables/2 times out).

Unfortunately, this code doesn't get a chance to run if mnesia itself refuses to
start (in many previous test runs the releases started--in some cases the
default table load algorithm worked just fine, and in other failure scenarios
the force_table_load was necessary--but the system always manged to start until
now).

Surely I must just be short on coffee (or sleep) or both. Any help would be
greatly appreciated.

-Rick


From jao@REDACTED  Wed Oct 17 06:42:36 2007
From: jao@REDACTED (Jack Orenstein)
Date: Wed, 17 Oct 2007 00:42:36 -0400
Subject: [erlang-questions] gen_server restarted by supervisor not working
	right
Message-ID: <7CDB9439-7310-4901-932F-BA58A8F64B75@geophile.com>

I've been going through various pieces of OTP documentation and trying
to put together my own simple code that demonstrates a supervisor
restarting a server.

I see the server starting and then crashing, (the test driver sends a
message that calls exit/1). I see the restart, but then the program
exits, apparently before the test driver's attempt to send a message
to the restarted server.

Here is the test driver, test.erl:

     -module(test).

     -export([main/0]).

     -include_lib("cluster.hrl").

     main() ->
         ?DUMP(cluster_supervisor:start_in_shell_for_testing()),
         ?DUMP(region_server:hello(world)),
         ?DUMP(region_server:crash()),
         ?DUMP(region_server:hello(again)).

The rest of the code appears below, but here is the test output:

     region_server:52 - "{ start_link , RegionNumber }" = {start_link,0}
     region_server:23 - "{ init , ? MODULE , RegionNumber }" = {init,
                                                                    
region_server,
                                                                   0}
     test:8 - "cluster_supervisor : start_in_shell_for_testing ( )" =  
true
     region_server:56 - "{ hello , X }" = {hello,world}
     region_server:30 - "{ hello , X }" = {hello,world}
     test:9 - "region_server : hello ( world )" = {hello,world,0}
     region_server:60 - "crash" = crash
     region_server:33 - "{ crash , RegionNumber }" = {crash,0}
     region_server:43 - "{ terminate , ? MODULE }" =  
{terminate,region_server}

     =ERROR REPORT==== 17-Oct-2007::00:30:41 ===
     ** Generic server region_server terminating
     ** Last message in was {crash}
     ** When Server state == 0
     ** Reason for termination ==
     ** {crash,0}
     region_server:52 - "{ start_link , RegionNumber }" = {start_link,0}
     region_server:23 - "{ init , ? MODULE , RegionNumber }" = {init,
                                                                    
region_server,
                                                                   0}
     {"init terminating in do_boot",{{crash,0},{gen_server,call, 
[region_server,{crash}]}}}

     Crash dump was written to: erl_crash.dump
     init terminating in do_boot ()

The output from region_server:hello(world), before the crash, appears;
but the output from region_server:hello(again), does not.

What am I doing wrong?

Jack Orenstein


Other modules:

Here is cluster_supervisor:

     -module(cluster_supervisor).
     -behavior(supervisor).

     -export([start/0,
              start_in_shell_for_testing/0,
              start_link/1,
              init/1]).

     -include_lib("cluster.hrl").

     start() ->
         spawn(fun() ->
                       supervisor:start_link({local, ?MODULE}, ? 
MODULE, [])
               end).

     start_in_shell_for_testing() ->
         {ok, P} = supervisor:start_link({local, ?MODULE}, ?MODULE, []),
         unlink(P).

     start_link(Args) ->
         supervisor:start_link({local, ?MODULE}, ?MODULE, Args).

     init([]) ->
         {ok, {{one_for_one, 3, 10},
               [{region0,
                 {region_server, start_link, [0]},
                 permanent,
                 10000,
                 worker,
                 [region_server]}
               ]}}.

And here is region_server:

     -module(region_server).
     -behavior(gen_server).

     -include_lib("cluster.hrl").

     %% gen_server api
     -export([init/1,
             handle_call/3,
             handle_cast/2,
             handle_info/2,
             terminate/2,
             code_change/3]).

     %% region_server api
     -export([start_link/1,
             hello/1,
             crash/0]).

     %% gen_server api

     init([RegionNumber]) ->
         process_flag(trap_exit, true),
         ?DUMP({init, ?MODULE, RegionNumber}),
         {ok, RegionNumber}.

     handle_call({region}, _From, RegionNumber) ->
         ?DUMP({region, RegionNumber}),
         {reply, {region, RegionNumber}, RegionNumber};
     handle_call({hello, X}, _From, RegionNumber) ->
         ?DUMP({hello, X}),
         {reply, {hello, X, RegionNumber}, RegionNumber};
     handle_call({crash}, _From, RegionNumber) ->
         ?DUMP({crash, RegionNumber}),
         exit({crash, RegionNumber}).

     handle_cast(_Cast, N) ->
         {noreply, N}.

     handle_info(_Info, N) ->
         {noreply, N}.

     terminate(_Readon, _N) ->
         ?DUMP({terminate, ?MODULE}),
         ok.

     code_change(_, N, _) ->
         {ok, N}.

     %% region_server api

     start_link(RegionNumber) ->
         ?DUMP({start_link, RegionNumber}),
         gen_server:start_link({local, ?MODULE}, ?MODULE,  
[RegionNumber], []).

     hello(X) ->
         ?DUMP({hello, X}),
         gen_server:call(?MODULE, {hello, X}).

     crash() ->
         ?DUMP(crash),
         gen_server:call(?MODULE, {crash}).

The DUMP macro is defined in cluster.hrl:

     -define(DUMP(X), io:format("~p:~p - ~p = ~p~n", [?MODULE, ? 
LINE, ??X, X])).	


From yinso.chen@REDACTED  Wed Oct 17 07:03:05 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 16 Oct 2007 22:03:05 -0700
Subject: [erlang-questions] *current* value?
Message-ID: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>

Hi all -

Is process dictionary the only built-in language level facility to return
context-dependent data, in Erlang?  e.g., self() returns the "current"
process.

%% hopefully only assign once...
set_current_request(Req) ->
  set(current_request, Req).

%% retrieve the request.
current_request() ->
  get(current_request).

Please let me know if there are better ways of accomplishing the goal.
Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071016/e18a674a/attachment.htm>

From bjarne@REDACTED  Wed Oct 17 08:07:23 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Wed, 17 Oct 2007 08:07:23 +0200
Subject: [erlang-questions] Erlang for iSGTW?
References: <3720.128.141.128.113.1192541124.squirrel@www.isgtw.org>
Message-ID: <002001c81084$2276d600$461469d4@Dell>

Hello Cristy,

I am sending this answer also to the Erlang questions list in the hope that 
someone will pick up this interesting thread!

Best wishes

Bjarne

----- Original Message ----- 
From: "Cristy Burne - iSGTW" <editor@REDACTED>
To: <euc@REDACTED>
Sent: Tuesday, October 16, 2007 3:25 PM
Subject: Erlang for iSGTW?


Hi,

I am interested in producing an article introducing Erlang and its
potential for distributed computing to readers of International Science
Grid This Week.

Can you please point me in the direction of someone suitable to contact?

Many thanks,

Cristy Burne
____________________________________
Cristy Burne, Editor
International Science Grid This Week
www.isgtw.org
+41 (0)22 76 75590 (Office)
+41 (0) 76 487 0486 (European mobile)
+1 630 398 0559 (US mobile)

Subscribe FREE at www.isgtw.org

____________________________________
Cristy Burne, Editor
International Science Grid This Week
www.isgtw.org
+41 (0)22 76 75590 (Office)
+41 (0) 76 487 0486 (European mobile)
+1 630 398 0559 (US mobile)

Subscribe FREE at www.isgtw.org




From erlang@REDACTED  Wed Oct 17 10:40:56 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Wed, 17 Oct 2007 10:40:56 +0200
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <9b08084c0710170140j7d5db4dfmcf145a180659e8f5@mail.gmail.com>

Erlang/OCaml/Haskell belong to the same language family - if you learn any one
of them then learning the next one in the family will be a lot easier
than starting from scratch.

These language differ - but have the same core concepts - the idea of immutable
state - programming with immutable state is the thing that you need to learn.
The details of how you do this vary from language to language (you can
use processes
with tail recursion to model state in Erlang, or monads in haskell, etc.).

I'd start with the language that most suits your problem domain - a
rough guess might
be to think of these languages as follows:

OCaml - use as a replacement for C - good for implementing virtual
machine emulators
tightly coded non-distributed applications.

Erlang - use as a replacement for Java - good for programming
distributed fault-tolerant
applications - good support for multicores/concurrency. Good as a glue
language to
glue together components co-ordinate activities on different machines etc.

Haskell - use for implementing domain specific languages, symbolic
computations etc.

Asking which is best is like asking which is best out of C++/java/C#
or perl/python/ruby

Languages come in 3-packs - learn any one then learning the others is
easy - similar ideas
different packaging.

/Joe Armstrong



On 10/16/07, Lone Wolf <hubaghdadi@REDACTED> wrote:
> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
>
>
>
> Deep into that darkness peering, long I stood there, wondering, fearing,
> Doubting, dreaming dreams no mortal ever dreamed before.
> E.A Poe
>
>
>
>  ________________________________
> Catch up on fall's hot new shows on Yahoo! TV. Watch previews, get listings,
> and more!
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From mvanier@REDACTED  Tue Oct 16 01:48:53 2007
From: mvanier@REDACTED (Michael Vanier)
Date: Mon, 15 Oct 2007 16:48:53 -0700
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <536569.99564.qm@web51106.mail.re2.yahoo.com>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
Message-ID: <4713FC65.4010404@cs.caltech.edu>

Learn both.

Mike

Lone Wolf wrote:
> Hi.
> Recently, I have been trying to learn a language other than Java.
> Erlang seems to be ready for the prime time (used in the commercial world).
> On the other side, OCaml seems to be a cool and capable language.
> I want to learn something useful and fun.
> Tell what do you think ?
> (if you are Erlang die-hard, please put this aside ;))
> 
> */Deep into that darkness peering, long I stood there, wondering, 
> fearing, Doubting, dreaming dreams no mortal ever dreamed before./*
> */E.A Poe/*
> *//*
> *//* 
> 
> ------------------------------------------------------------------------
> Catch up on fall's hot new shows 
> <http://us.rd.yahoo.com/tv/mail/tagline/falltv/evt=47093/*http://tv.yahoo.com/collections/3658 
>  > on Yahoo! TV. Watch previews, get listings, and more!
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From hakan@REDACTED  Wed Oct 17 11:32:14 2007
From: hakan@REDACTED (Hakan Mattsson)
Date: Wed, 17 Oct 2007 11:32:14 +0200 (CEST)
Subject: [erlang-questions] why might mnesia:start() hang?
In-Reply-To: <20071017040536.GB19639@vailsys.com>
References: <20071017040536.GB19639@vailsys.com>
Message-ID: <Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>


There may be several causes for this to happen:

- It may be the case that some other application has
  encountered a deadlock in its startup. This may for
  example occur if that application is invoking functions
  in the 'application' API during its startup. It may also
  occur if a process dies during the application
  startup. Then its supervisor will not restart the
  process until it has started all its children. 

- It could also be that it is Mnesia that refuses to
  start. This may happen if the system first crashes
  during the critical phase in transaction commit and one
  of the other nodes does not come up again. Then Mnesia
  will by default wait indefinitely for the other node to
  be available before it finishes its own startup. See
  the documentation about the Mnesia parameter
  max_wait_for_decision for more info. If you set the
  Mnesia debug level to at least 'verbose' (before you
  start Mnesia) you will get a printout when this happens.

- ...

/H?kan


On Tue, 16 Oct 2007, Rick Pettit wrote:

RP> Date: Tue, 16 Oct 2007 23:05:36 -0500
RP> From: Rick Pettit <rpettit@REDACTED>
RP> To: erlang-questions@REDACTED
RP> Subject: [erlang-questions] why might mnesia:start() hang?
RP> 
RP> I seem to have encountered a situation in which I am
RP> unable to start mnesia.  Attempts to start mnesia (via
RP> mnesia:start/0) hang the erlang shell.
RP> 
RP> In the scenario below there are 2 physical servers,
RP> each running an instance of the foo_rel and
RP> bar_rel. The second physical server,
RP> someother.somedomain, has been halted prior to starting
RP> the nodes somebox.somedomain.
RP> 
RP> The foo_rel instances contain disc_copy tables--bar_rel
RP> instances contain ram_copies only.
RP> 
RP> (foo_rel@REDACTED)1> application:which_applications().
RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
RP> 
RP> 
RP>    NOTE: there are other applications in this release which *should* be running
RP>          but are not, almost certainly due to the fact that mnesia is refusing
RP>          to start
RP> 
RP> 
RP> (foo_rel@REDACTED)2> mnesia:info().
RP> ===> System info in version "4.3.5", debug level = none <===
RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
RP> use fallback at restart = false
RP> running db nodes   = ['foo_rel@REDACTED']
RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
RP> ok
RP> 
RP> 
RP> (foo_rel@REDACTED)3> mnesia:stop().
RP> stopped
RP> 
RP> 
RP> (foo_rel@REDACTED)4> mnesia:start().
RP> ...shell hangs forever...
RP> 
RP> 
RP> Shell back into the node, try again:
RP> 
RP> 
RP> (foo_rel@REDACTED)1> mnesia:info().
RP> ===> System info in version "4.3.5", debug level = none <===
RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
RP> use fallback at restart = false
RP> running db nodes   = ['foo_rel@REDACTED']
RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
RP> ok
RP> 
RP> 
RP> (foo_rel@REDACTED)2> application:which_applications().
RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
RP> 
RP> 
RP> (foo_rel@REDACTED)3> mnesia:start().
RP> ...hangs forever...
RP> 
RP> 
RP> ======
RP> 
RP> I cannot seem to figure out:
RP> 
RP>   1) why mnesia refuses to start
RP> 
RP>   2) why mnesia:start() hangs forever at the shell
RP>     (vs. return an error, etc)
RP> 
RP> Any applications requiring mnesia tables do a
RP> mnesia:wait_for_tables/2 on them.
RP> 
RP> A special process performs a mnesia:force_load_table/1,
RP> if necessary (e.g. when wait_for_tables/2 times out).
RP> 
RP> Unfortunately, this code doesn't get a chance to run if
RP> mnesia itself refuses to start (in many previous test
RP> runs the releases started--in some cases the default
RP> table load algorithm worked just fine, and in other
RP> failure scenarios the force_table_load was
RP> necessary--but the system always manged to start until
RP> now).
RP> 
RP> Surely I must just be short on coffee (or sleep) or
RP> both. Any help would be greatly appreciated.
RP> 
RP> -Rick
RP> _______________________________________________
RP> erlang-questions mailing list
RP> erlang-questions@REDACTED
RP> http://www.erlang.org/mailman/listinfo/erlang-questions

From ft@REDACTED  Tue Oct 16 10:02:28 2007
From: ft@REDACTED (Fredrik Thulin)
Date: Tue, 16 Oct 2007 10:02:28 +0200
Subject: [erlang-questions] Basic questions,
 also related to yaws and mnesia
In-Reply-To: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
References: <df27a55b0710151734i872e0e7kf05b2c71d543c3b5@mail.gmail.com>
Message-ID: <47147014.5090700@it.su.se>

Berlin Brown wrote:
> I should probably move away from yaws for a sec and go over the
> basics, but I am really close to what I want to do.
> 
> I have a record and a list of records, but I don't know the best ways
> to extract the data and show the data out of the record.
> 
> For example, with this code; how would I iterate(map?) those links and
> display them.
> 
> out(Args) ->
> 	{ok, EntityLinks} = botlist_find_links:find_links(),
> 	{ html,
> 	 	yaws_api:f("~p", [EntityLinks])
> 	}.
> 
> This is the record.
> -record(entity_links, {main_url, url_title, keywords, rating,
> full_name, created_on}).
> 

If I understand correctly what you want to do, try something like this :

out(Args) ->
     {ok, EntityLinks} = botlist_find_links:find_links(),
     {html, [
         [pretty_print(Entity) || Entity <- EntityLinks]
     ]}.

pretty_print(Entity) ->
     {p, [], [Entity#entity_links.main_url]}.

Totally untested. This will call pretty_print/1 once for each entry in 
the EntityLinks list, so that you can extract whatever information you 
want from each of the entity_link records.

/Fredrik


From saleyn@REDACTED  Wed Oct 17 14:42:31 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 17 Oct 2007 07:42:31 -0500
Subject: [erlang-questions] code_server patch
In-Reply-To: <4715F013.4010400@gmail.com>
References: <4715F013.4010400@gmail.com>
Message-ID: <47160337.4070601@gmail.com>

Sorry I realized that in my previous email I included a wrong version of 
the patch.  This one is for R11B-5 and does the proper ordering of 
directory inclusions.

Serge

Serge Aleynikov wrote:
> Hi,
> 
> In the current installation of Erlang I ran into a need to separate the 
> distribution installation and various 3rd party libraries, so that at 
> startup Erlang's code server still was able to auto-locate all libraries 
> under a given set of paths.  Unfortunately by default it only examines 
> the content of $ROOT/lib path, and the rest needs to be provided via 
> "-pa" or "-pz" command-line options.
> 
> I came up with this little patch to the code_server that enables to 
> specify the environment variable ERL_LIBS that can contain a delimited 
> list of paths that would be searched for "*/ebin" sub-dirs and if found, 
> included in the library path.  As a result you can install Erlang in one 
> location and conveniently maintain your libraries in another location:
> 
> Example usage:
> 
> $ export ERL_LIBS=~/opt/erlang/lib:/opt/erlang/lib
> $ erl
> 1> lists:reverse(code:get_path()).
> 
> I verified that this patch works for both erl and erlc.
> 
> Are there any drawbacks in including this patch to the distribution?
> 
> Serge

-------------- next part --------------
A non-text attachment was scrubbed...
Name: code_server.R11B-5.patch
Type: application/octet-stream
Size: 1705 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/9c408a1a/attachment.obj>

From saleyn@REDACTED  Wed Oct 17 13:20:51 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 17 Oct 2007 06:20:51 -0500
Subject: [erlang-questions] code_server patch
Message-ID: <4715F013.4010400@gmail.com>

Hi,

In the current installation of Erlang I ran into a need to separate the 
distribution installation and various 3rd party libraries, so that at 
startup Erlang's code server still was able to auto-locate all libraries 
under a given set of paths.  Unfortunately by default it only examines 
the content of $ROOT/lib path, and the rest needs to be provided via 
"-pa" or "-pz" command-line options.

I came up with this little patch to the code_server that enables to 
specify the environment variable ERL_LIBS that can contain a delimited 
list of paths that would be searched for "*/ebin" sub-dirs and if found, 
included in the library path.  As a result you can install Erlang in one 
location and conveniently maintain your libraries in another location:

Example usage:

$ export ERL_LIBS=~/opt/erlang/lib:/opt/erlang/lib
$ erl
1> lists:reverse(code:get_path()).

I verified that this patch works for both erl and erlc.

Are there any drawbacks in including this patch to the distribution?

Serge
-------------- next part --------------
A non-text attachment was scrubbed...
Name: code_server.R11B-4.patch
Type: application/octet-stream
Size: 1705 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/6f7c9c14/attachment.obj>

From ola.a.andersson@REDACTED  Wed Oct 17 14:58:03 2007
From: ola.a.andersson@REDACTED (Ola Andersson A)
Date: Wed, 17 Oct 2007 14:58:03 +0200
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
Message-ID: <148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>

Normally you never use the process dictionary. You could use a named ets
table instead but there are very very few
situations where you actually need either of those.
Most of the time there is some sort of main loop where the code is
executed. The input data to the loop function
is the place to use as storage for any data you need to keep. 
/OLA.

 

________________________________

	From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of YC
	Sent: den 17 oktober 2007 07:03
	To: Erlang Questions
	Subject: [erlang-questions] *current* value?
	
	
	Hi all - 
	
	Is process dictionary the only built-in language level facility
to return context-dependent data, in Erlang?  e.g., self() returns the
"current" process.
	
	%% hopefully only assign once...
	set_current_request(Req) ->
	  set(current_request, Req).
	
	%% retrieve the request. 
	current_request() ->
	  get(current_request).
	
	Please let me know if there are better ways of accomplishing the
goal.
	Thanks,
	yc
	
	
	
	

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/3e717ca6/attachment.htm>

From jay@REDACTED  Wed Oct 17 15:09:37 2007
From: jay@REDACTED (Jay Nelson)
Date: Wed, 17 Oct 2007 06:09:37 -0700
Subject: [erlang-questions] More nodes than machines
Message-ID: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>

This question is directed to Bob Ippolito because he said the  
following in the thread on "OCaml vs Erlang":

 > A year later we have about 16 machines running 80 Erlang nodes  
powering about 16
 > different "components" of our infrastructure

1) Why did you choose to use multiple nodes on each physical machine?

     - Are your machines multicore?
     - Are you using SMP for your erlang nodes?
     - Are you using virtualized machines (via Xen, VMWare or other  
means)?
     - What kind of machines are you running (CPU and memory)?

2) Any performance numbers on simultaneous connections or connection  
rate per minute sustainable by your mochiweb http server?


jay



From kevin@REDACTED  Wed Oct 17 16:37:30 2007
From: kevin@REDACTED (Kevin A. Smith)
Date: Wed, 17 Oct 2007 10:37:30 -0400
Subject: [erlang-questions] More nodes than machines
In-Reply-To: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>
References: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>
Message-ID: <D398AD75-56E6-4905-BD02-BC91DA1804C2@hypotheticalabs.com>

Bob was kind enough to answer many of these questions for me which  
I've blogged about here:

http://weblog.hypotheticalabs.com/?p=171

--Kevin
On Oct 17, 2007, at 9:09 AM, Jay Nelson wrote:

> This question is directed to Bob Ippolito because he said the
> following in the thread on "OCaml vs Erlang":
>
>> A year later we have about 16 machines running 80 Erlang nodes
> powering about 16
>> different "components" of our infrastructure
>
> 1) Why did you choose to use multiple nodes on each physical machine?
>
>      - Are your machines multicore?
>      - Are you using SMP for your erlang nodes?
>      - Are you using virtualized machines (via Xen, VMWare or other
> means)?
>      - What kind of machines are you running (CPU and memory)?
>
> 2) Any performance numbers on simultaneous connections or connection
> rate per minute sustainable by your mochiweb http server?
>
>
> jay
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From rpettit@REDACTED  Wed Oct 17 16:49:01 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Wed, 17 Oct 2007 09:49:01 -0500
Subject: [erlang-questions] why might mnesia:start() hang?
In-Reply-To: <Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
References: <20071017040536.GB19639@vailsys.com>
	<Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
Message-ID: <20071017144901.GA26511@vailsys.com>

On Wed, Oct 17, 2007 at 11:32:14AM +0200, Hakan Mattsson wrote:
> 
> There may be several causes for this to happen:
> 
> - It may be the case that some other application has
>   encountered a deadlock in its startup. This may for
>   example occur if that application is invoking functions
>   in the 'application' API during its startup. It may also
>   occur if a process dies during the application
>   startup. Then its supervisor will not restart the
>   process until it has started all its children. 

I will check but don't believe this is my case.

> - It could also be that it is Mnesia that refuses to
>   start. This may happen if the system first crashes
>   during the critical phase in transaction commit and one
>   of the other nodes does not come up again. Then Mnesia
>   will by default wait indefinitely for the other node to
>   be available before it finishes its own startup.

This is what I suspect happened, just don't know the details (yet :-).

>   See
>   the documentation about the Mnesia parameter
>   max_wait_for_decision for more info.

I stumbled on this last night but have not yet gotten a chance to 
try it--thanks.

> If you set the
>   Mnesia debug level to at least 'verbose' (before you
>   start Mnesia) you will get a printout when this happens.

I'll try that, too.

Thanks again for all your help,

-Rick

> On Tue, 16 Oct 2007, Rick Pettit wrote:
> 
> RP> Date: Tue, 16 Oct 2007 23:05:36 -0500
> RP> From: Rick Pettit <rpettit@REDACTED>
> RP> To: erlang-questions@REDACTED
> RP> Subject: [erlang-questions] why might mnesia:start() hang?
> RP> 
> RP> I seem to have encountered a situation in which I am
> RP> unable to start mnesia.  Attempts to start mnesia (via
> RP> mnesia:start/0) hang the erlang shell.
> RP> 
> RP> In the scenario below there are 2 physical servers,
> RP> each running an instance of the foo_rel and
> RP> bar_rel. The second physical server,
> RP> someother.somedomain, has been halted prior to starting
> RP> the nodes somebox.somedomain.
> RP> 
> RP> The foo_rel instances contain disc_copy tables--bar_rel
> RP> instances contain ram_copies only.
> RP> 
> RP> (foo_rel@REDACTED)1> application:which_applications().
> RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> RP> 
> RP> 
> RP>    NOTE: there are other applications in this release which *should* be running
> RP>          but are not, almost certainly due to the fact that mnesia is refusing
> RP>          to start
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)2> mnesia:info().
> RP> ===> System info in version "4.3.5", debug level = none <===
> RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> RP> use fallback at restart = false
> RP> running db nodes   = ['foo_rel@REDACTED']
> RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> RP> ok
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)3> mnesia:stop().
> RP> stopped
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)4> mnesia:start().
> RP> ...shell hangs forever...
> RP> 
> RP> 
> RP> Shell back into the node, try again:
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)1> mnesia:info().
> RP> ===> System info in version "4.3.5", debug level = none <===
> RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> RP> use fallback at restart = false
> RP> running db nodes   = ['foo_rel@REDACTED']
> RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> RP> ok
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)2> application:which_applications().
> RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)3> mnesia:start().
> RP> ...hangs forever...
> RP> 
> RP> 
> RP> ======
> RP> 
> RP> I cannot seem to figure out:
> RP> 
> RP>   1) why mnesia refuses to start
> RP> 
> RP>   2) why mnesia:start() hangs forever at the shell
> RP>     (vs. return an error, etc)
> RP> 
> RP> Any applications requiring mnesia tables do a
> RP> mnesia:wait_for_tables/2 on them.
> RP> 
> RP> A special process performs a mnesia:force_load_table/1,
> RP> if necessary (e.g. when wait_for_tables/2 times out).
> RP> 
> RP> Unfortunately, this code doesn't get a chance to run if
> RP> mnesia itself refuses to start (in many previous test
> RP> runs the releases started--in some cases the default
> RP> table load algorithm worked just fine, and in other
> RP> failure scenarios the force_table_load was
> RP> necessary--but the system always manged to start until
> RP> now).
> RP> 
> RP> Surely I must just be short on coffee (or sleep) or
> RP> both. Any help would be greatly appreciated.
> RP> 
> RP> -Rick
> RP> _______________________________________________
> RP> erlang-questions mailing list
> RP> erlang-questions@REDACTED
> RP> http://www.erlang.org/mailman/listinfo/erlang-questions


From opendev@REDACTED  Wed Oct 17 18:08:58 2007
From: opendev@REDACTED (Joern)
Date: Wed, 17 Oct 2007 18:08:58 +0200
Subject: [erlang-questions] Multiple gen_fsm instances and gen_tcp:accept/1
Message-ID: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>

Dear erlang-questions,

I'd like to spawn a supervised instance of (the same) gen_fsm instance
for each successful return of gen_tcp:accept/1 - these non-permanent
gen_fsm's talk a stateful protocol over TCP / IP.

However I am a little bit unsure how to implement this right: use a
gen_server with a custom receive loop (I cannot use active receiving,
the protocol uses yet another funny framing) that uses proc_lib to
instantiate each gen_fsm? If so, how would I specify the appropriate
supervision tree?

Best regards,


Joern
--


From vances@REDACTED  Wed Oct 17 18:45:46 2007
From: vances@REDACTED (Vance Shipley)
Date: Wed, 17 Oct 2007 12:45:46 -0400
Subject: [erlang-questions] Multiple gen_fsm instances
	and	gen_tcp:accept/1
In-Reply-To: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>
References: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>
Message-ID: <20071017164517.GA462@little-black-book.motivity.ca>

On Wed, Oct 17, 2007 at 06:08:58PM +0200, Joern wrote:
}  [...] uses proc_lib to instantiate each gen_fsm? If so,
}  how would I specify the appropriate supervision tree?


Joern,

Create a simple_one_for_one supervisor.  Then start the workers
with supervisor:start_child/2.

	-Vance


From opendev@REDACTED  Wed Oct 17 19:09:51 2007
From: opendev@REDACTED (Joern)
Date: Wed, 17 Oct 2007 19:09:51 +0200
Subject: [erlang-questions] Multiple gen_fsm instances and
	gen_tcp:accept/1
In-Reply-To: <20071017164517.GA462@little-black-book.motivity.ca>
References: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>
	<20071017164517.GA462@little-black-book.motivity.ca>
Message-ID: <9e009ad0710171009n96e725bif056a6dbd33f4ca4@mail.gmail.com>

Thanks tty, thanks Vance,

On 10/17/07, Vance Shipley <vances@REDACTED> wrote:

> On Wed, Oct 17, 2007 at 06:08:58PM +0200, Joern wrote:

> Create a simple_one_for_one supervisor.  Then start the workers
> with supervisor:start_child/2.

Seems right at first glance. Just to sketch it out: I write a
gen_server which starts in an infinite loop. It adds instances of the
gen_fsm each time gen_tcp:accept/1 returns to it's sibling in the
supervision tree, a simple_one_for_one supervisor. Am I correct?

Another (unrelated) thought: for such a simple case I do not need a
gen_server, a simple supervisable process should do. Is there a reason
why there is not a behaviour such as e.g. 'plain' for such cases in
the OTP?

Best regards,


Joern
--


From bbrown@REDACTED  Wed Oct 17 20:01:59 2007
From: bbrown@REDACTED (bbrown)
Date: Wed, 17 Oct 2007 14:01:59 -0400
Subject: [erlang-questions] Simple, how to do "paging" with mnesia records
Message-ID: <20071017175623.M86870@www.botspiritcompany.com>

For example, in java I might do the following to set at some "start" index 
and then only return X number of records.  This may operate independent of 
the ids in question.

query.setFirstResult(page * pageSize);
query.setMaxResults(pageSize);
List data = query.list();


What is a mnesia approach for doing this.

Eg, I was going to use something along the following And then maybe do a less 
than link_id > than.  But I havent made it yet.  And yes, I probably need to 
do a little bit more research.  But I thought I would ask before I waste of 
day of work.

page_links() ->
    Q = query [E || E <- mnesia:table(entity_links),
		    E <- [#entity_links{link_id=1},
			  #entity_links{link_id=2}
			 ]]
	end,
    F = fun() -> mnemosyne:eval(Q) end,
    case mnesia:transaction(F) of
        {atomic, Result}  -> {ok, Result};
        {aborted, Reason} -> {error, Reason}
    end.



--
Berlin Brown
[berlin dot brown at gmail dot com]
http://botspiritcompany.com/botlist/?



From yinso.chen@REDACTED  Wed Oct 17 20:10:37 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 11:10:37 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
Message-ID: <779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>

On 10/17/07, Ola Andersson A <ola.a.andersson@REDACTED> wrote:
>
>  > Normally you never use the process dictionary. You could use a named
> ets table instead
>

Ets feels rather heavy for what I'm thinking about (see below).  What's the
pro/con between ets and process dictionary?

> but there are very very few
> > situations where you actually need either of those.
>

What I'm trying to figure out is if there are language support to access
variables that are not passed in, and hence "current", i.e. self() is the
function that returns the address of the current process, without it having
to be on the signature of every function that needs it.

> Most of the time there is some sort of main loop where the code is
> executed. The input data to the > loop function
> > is the place to use as storage for any data you need to keep.
>

I'm not sure if I follow.  Are you saying that the variable within the main
loop can be accessed directly from the worker function without being passed
in?

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/20090839/attachment.htm>

From dking@REDACTED  Wed Oct 17 20:34:33 2007
From: dking@REDACTED (David King)
Date: Wed, 17 Oct 2007 11:34:33 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
Message-ID: <22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>

>> Most of the time there is some sort of main loop where the code is  
>> executed. The input data to the > loop function
>> is the place to use as storage for any data you need to keep.
> I'm not sure if I follow.  Are you saying that the variable within  
> the main loop can be accessed directly from the worker function  
> without being passed in?

No, he's saying that usually you write programs like this:

main_loop(State) ->
   receive
       {eat_cookies,Cookies} ->
         NewState=eat_cookies(State,Cookies);
       {bake_cookies} ->
         NewState=bake_cookies(State)
     end,
   main_loop(NewState).

So that all of the state is encased in the arguments, rather than  
putting some in the process dictionary.


From yinso.chen@REDACTED  Wed Oct 17 20:56:22 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 11:56:22 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
Message-ID: <779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>

On 10/17/07, David King <dking@REDACTED> wrote:
>
> > No, he's saying that usually you write programs like this:
>
> > So that all of the state is encased in the arguments, rather than
> > putting some in the process dictionary.
>

Okay, that I understand - process dictionary is discouraged in Erlang, Joe
states so in his book.  He did, however, says this in the book:

One form of usage that I do approve of is to use the processes dictionary to
store "write-once" variables.

Anyhow - I'm simply trying to see if there are other ways besides process
dictionary to simulate "current" variables.  Seems like there is not.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/80c278d3/attachment.htm>

From kg9020@REDACTED  Wed Oct 17 20:59:10 2007
From: kg9020@REDACTED (kg9020@REDACTED)
Date: Wed, 17 Oct 2007 13:59:10 -0500
Subject: [erlang-questions] blue tail ticket tracker
Message-ID: <96fd17a20710171159u6a805dc4w4db59fe58a02e040@mail.gmail.com>

I am unable to get btt to run on ubuntu 4.1.2  linux 2.6.20-16 - generic .
Please look at the following error out put in attached file.  All guidance
will be welcome

  Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/7f60c349/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: btt-run.txt
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/7f60c349/attachment.txt>

From vances@REDACTED  Wed Oct 17 21:05:06 2007
From: vances@REDACTED (Vance Shipley)
Date: Wed, 17 Oct 2007 15:05:06 -0400
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171148w6cbafd4x9d38055b4cf5384a@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<20071017181937.GA643@little-black-book.motivity.ca>
	<779bf2730710171148w6cbafd4x9d38055b4cf5384a@mail.gmail.com>
Message-ID: <20071017190505.GC643@little-black-book.motivity.ca>

YC,

As I see it the process dictionary has one and only one
legitimate purpose and that is to store process data for
OTP tools such as the debugger.  If you are building a 
library to add tool functions to arbritray code modules
than using the process dictionary may be reasonable.

If you need a variable to persist through function calls
you would normally pass it as a function argument.  If
there are too many consider storing variable data in records
and pass the record (i.e. #state{a=1, b=2, ...}).

If there is too much process data for that approach look
to using ets.  Beyond that look to mnesia.

For a purely functional solution use a seperate process to
store the data and use message passing.

	-Vance


From rpettit@REDACTED  Wed Oct 17 21:58:55 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Wed, 17 Oct 2007 14:58:55 -0500
Subject: [erlang-questions] why might mnesia:start() hang?
In-Reply-To: <Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
References: <20071017040536.GB19639@vailsys.com>
	<Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
Message-ID: <20071017195855.GB26511@vailsys.com>

On Wed, Oct 17, 2007 at 11:32:14AM +0200, Hakan Mattsson wrote:
> 
> There may be several causes for this to happen:
> 
> - It may be the case that some other application has
>   encountered a deadlock in its startup. This may for
>   example occur if that application is invoking functions
>   in the 'application' API during its startup. It may also
>   occur if a process dies during the application
>   startup. Then its supervisor will not restart the
>   process until it has started all its children. 
> 
> - It could also be that it is Mnesia that refuses to
>   start. This may happen if the system first crashes
>   during the critical phase in transaction commit and one
>   of the other nodes does not come up again. Then Mnesia
>   will by default wait indefinitely for the other node to
>   be available before it finishes its own startup. See
>   the documentation about the Mnesia parameter
>   max_wait_for_decision for more info. If you set the
>   Mnesia debug level to at least 'verbose' (before you
>   start Mnesia) you will get a printout when this happens.

I have restarted the sick node with -mnesia debug true, and now I see (among
other things) in syslog output:

Oct 17 14:39:18 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 17487ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction {tid,33072,<0.118.0>} calling #Fun<mnesia_schema.33.14280620> with [] failed: 
Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug]  {aborted,{no_exists,some_table}}
Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction {tid,33071,<0.117.0>} calling #Fun<mnesia_schema.15.92354373> with [] failed: 
Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug]  {aborted,{no_exists,some_table}}


I suspect that some_table (which is, er, was, a disc_copy table) was deleted on
the primary node while the secondary was down (it is still down).

At this point it would be sufficient to get mnesia running at all on the
primary node. I've tried erasing all notion of some_table from the schema,
but without luck:

(foo_rel@REDACTED)2> mnesia:delete_table(some_table).                 
{aborted,{no_exists,some_table}}
(foo_rel@REDACTED)1> mnesia:del_table_copy(schema,'foo_rel@REDACTED'). 
{aborted,{no_exists,some_table}}

Clearly the deletion of some_table has really upset mnesia, because I keep
seeing the following repeated over and over:

Oct 17 14:56:51 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 31404ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:57:23 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 31412ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:57:54 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32007ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:58:26 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32441ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:58:59 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32886ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 14:59:07 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction log dump initiated by time_threshold: already_dumped
Oct 17 14:59:32 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 33081ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 15:00:05 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 33459ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 15:00:38 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 33699ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 15:01:12 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 33966ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 15:01:46 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 34583ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
Oct 17 15:02:07 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction log dump initiated by time_threshold: already_dumped
Oct 17 15:02:20 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 34827ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}

I have not yet tried to restart the node w/ -mnesia_wait_for_decision Timeout--
I was hoping to gain a better understanding of the current situation first.

I think at this point it would be sufficient to get mnesia to forget all 
about some_table. :-)

-Rick

> On Tue, 16 Oct 2007, Rick Pettit wrote:
> 
> RP> Date: Tue, 16 Oct 2007 23:05:36 -0500
> RP> From: Rick Pettit <rpettit@REDACTED>
> RP> To: erlang-questions@REDACTED
> RP> Subject: [erlang-questions] why might mnesia:start() hang?
> RP> 
> RP> I seem to have encountered a situation in which I am
> RP> unable to start mnesia.  Attempts to start mnesia (via
> RP> mnesia:start/0) hang the erlang shell.
> RP> 
> RP> In the scenario below there are 2 physical servers,
> RP> each running an instance of the foo_rel and
> RP> bar_rel. The second physical server,
> RP> someother.somedomain, has been halted prior to starting
> RP> the nodes somebox.somedomain.
> RP> 
> RP> The foo_rel instances contain disc_copy tables--bar_rel
> RP> instances contain ram_copies only.
> RP> 
> RP> (foo_rel@REDACTED)1> application:which_applications().
> RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> RP> 
> RP> 
> RP>    NOTE: there are other applications in this release which *should* be running
> RP>          but are not, almost certainly due to the fact that mnesia is refusing
> RP>          to start
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)2> mnesia:info().
> RP> ===> System info in version "4.3.5", debug level = none <===
> RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> RP> use fallback at restart = false
> RP> running db nodes   = ['foo_rel@REDACTED']
> RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> RP> ok
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)3> mnesia:stop().
> RP> stopped
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)4> mnesia:start().
> RP> ...shell hangs forever...
> RP> 
> RP> 
> RP> Shell back into the node, try again:
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)1> mnesia:info().
> RP> ===> System info in version "4.3.5", debug level = none <===
> RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> RP> use fallback at restart = false
> RP> running db nodes   = ['foo_rel@REDACTED']
> RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> RP> ok
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)2> application:which_applications().
> RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> RP> 
> RP> 
> RP> (foo_rel@REDACTED)3> mnesia:start().
> RP> ...hangs forever...
> RP> 
> RP> 
> RP> ======
> RP> 
> RP> I cannot seem to figure out:
> RP> 
> RP>   1) why mnesia refuses to start
> RP> 
> RP>   2) why mnesia:start() hangs forever at the shell
> RP>     (vs. return an error, etc)
> RP> 
> RP> Any applications requiring mnesia tables do a
> RP> mnesia:wait_for_tables/2 on them.
> RP> 
> RP> A special process performs a mnesia:force_load_table/1,
> RP> if necessary (e.g. when wait_for_tables/2 times out).
> RP> 
> RP> Unfortunately, this code doesn't get a chance to run if
> RP> mnesia itself refuses to start (in many previous test
> RP> runs the releases started--in some cases the default
> RP> table load algorithm worked just fine, and in other
> RP> failure scenarios the force_table_load was
> RP> necessary--but the system always manged to start until
> RP> now).
> RP> 
> RP> Surely I must just be short on coffee (or sleep) or
> RP> both. Any help would be greatly appreciated.
> RP> 
> RP> -Rick
> RP> _______________________________________________
> RP> erlang-questions mailing list
> RP> erlang-questions@REDACTED
> RP> http://www.erlang.org/mailman/listinfo/erlang-questions


From ulf.wiger@REDACTED  Wed Oct 17 22:07:52 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Wed, 17 Oct 2007 22:07:52 +0200
Subject: [erlang-questions] Multiple gen_fsm instances and
	gen_tcp:accept/1
In-Reply-To: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>
References: <9e009ad0710170908ge1585a3h761c820b115d1f74@mail.gmail.com>
Message-ID: <47166B98.4080202@ericsson.com>

Joern wrote:
> 
> However I am a little bit unsure how to implement this right: use a
> gen_server with a custom receive loop (I cannot use active receiving,
> the protocol uses yet another funny framing) 

Funny framing can be handled by a "middle-man process", which
simply sits between your FSM and the socket, stripping away
the framing. This way, your FSM can deal with nicely formatted
messages. In Erlang, middle-man processes add very little overhead.

BR,
Ulf W


From yinso.chen@REDACTED  Wed Oct 17 21:34:47 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 12:34:47 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <20071017190505.GC643@little-black-book.motivity.ca>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<20071017181937.GA643@little-black-book.motivity.ca>
	<779bf2730710171148w6cbafd4x9d38055b4cf5384a@mail.gmail.com>
	<20071017190505.GC643@little-black-book.motivity.ca>
Message-ID: <779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>

Hi Vance -

Thanks for the explanations for the different approaches in Erlang - this is
great.

Hopefully this follow-up doesn't make me sound like a troll - but I can't
help but notice that ets and even Mnesia is recommended before process
dictionary (which is very heavily frowned upon) in this thread, and I am not
sure if I understand the reason, unless process dictionary is technically
inferior or broken.  From FP perspective, neither is pure, so why one better
than another?

There must be a reason for the level of discouragement about a built-in
facility.

If this is a trite question - I apologize in advance.

Thanks,
yc

On 10/17/07, Vance Shipley < vances@REDACTED> wrote:
>
> YC,
>
> As I see it the process dictionary has one and only one
> legitimate purpose and that is to store process data for
> OTP tools such as the debugger.  If you are building a
> library to add tool functions to arbritray code modules
> than using the process dictionary may be reasonable.
>
> If you need a variable to persist through function calls
> you would normally pass it as a function argument.  If
> there are too many consider storing variable data in records
> and pass the record (i.e. #state{a=1, b=2, ...}).
>
> If there is too much process data for that approach look
> to using ets.  Beyond that look to mnesia.
>
> For a purely functional solution use a seperate process to
> store the data and use message passing.
>
>         -Vance
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/fb201aaf/attachment.htm>

From rpettit@REDACTED  Wed Oct 17 22:30:00 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Wed, 17 Oct 2007 15:30:00 -0500
Subject: [erlang-questions] why might mnesia:start() hang?
In-Reply-To: <20071017195855.GB26511@vailsys.com>
References: <20071017040536.GB19639@vailsys.com>
	<Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
	<20071017195855.GB26511@vailsys.com>
Message-ID: <20071017203000.GA16205@vailsys.com>

On Wed, Oct 17, 2007 at 02:58:55PM -0500, Rick Pettit wrote:
> On Wed, Oct 17, 2007 at 11:32:14AM +0200, Hakan Mattsson wrote:
> > 
> > There may be several causes for this to happen:
> > 
> > - It may be the case that some other application has
> >   encountered a deadlock in its startup. This may for
> >   example occur if that application is invoking functions
> >   in the 'application' API during its startup. It may also
> >   occur if a process dies during the application
> >   startup. Then its supervisor will not restart the
> >   process until it has started all its children. 
> > 
> > - It could also be that it is Mnesia that refuses to
> >   start. This may happen if the system first crashes
> >   during the critical phase in transaction commit and one
> >   of the other nodes does not come up again. Then Mnesia
> >   will by default wait indefinitely for the other node to
> >   be available before it finishes its own startup. See
> >   the documentation about the Mnesia parameter
> >   max_wait_for_decision for more info. If you set the
> >   Mnesia debug level to at least 'verbose' (before you
> >   start Mnesia) you will get a printout when this happens.
> 
> I have restarted the sick node with -mnesia debug true, and now I see (among
> other things) in syslog output:
> 
> Oct 17 14:39:18 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 17487ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction {tid,33072,<0.118.0>} calling #Fun<mnesia_schema.33.14280620> with [] failed: 
> Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug]  {aborted,{no_exists,some_table}}
> Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction {tid,33071,<0.117.0>} calling #Fun<mnesia_schema.15.92354373> with [] failed: 
> Oct 17 14:39:31 somebox.somedomain foo1: [ID 702911 local6.debug]  {aborted,{no_exists,some_table}}
> 
> 
> I suspect that some_table (which is, er, was, a disc_copy table) was deleted on
> the primary node while the secondary was down (it is still down).
> 
> At this point it would be sufficient to get mnesia running at all on the
> primary node. I've tried erasing all notion of some_table from the schema,
> but without luck:
> 
> (foo_rel@REDACTED)2> mnesia:delete_table(some_table).                 
> {aborted,{no_exists,some_table}}
> (foo_rel@REDACTED)1> mnesia:del_table_copy(schema,'foo_rel@REDACTED'). 
> {aborted,{no_exists,some_table}}

Here's some additional information that might help (e.g. I see some_table as
a member of the local_tables list--that's probably not good if mnesia doesn't
think the table exists anymore :-)

(foo_rel@REDACTED)5> mnesia:table_info(schema,all).
[{access_mode,read_write},
 {active_replicas,['foo_rel@REDACTED']},
 {arity,3},
 {attributes,[table,cstruct]},
 {checkpoints,[]},
 {commit_work,[{subscribers,[<0.54.0>],[]}]},
 {cookie,{{1187,632480,715351},'foo_rel@REDACTED'}},
 {cstruct,{cstruct,schema,
                   set,
                   ['bar_rel@REDACTED','bar_rel@REDACTED'],
                   ['foo_rel@REDACTED','foo_rel@REDACTED'],
                   [],
                   0,
                   read_write,
                   [],
                   [],
                   false,
                   schema,
                   [table,cstruct],
                   [],
                   [],
                   {{1187,632480,...},'foo_rel@REDACTED'},
                   {{34,...},{...}}}},
 {disc_copies,['foo_rel@REDACTED','foo_rel@REDACTED']},
 {disc_only_copies,[]},
 {frag_properties,[]},
 {index,[]},
 {load_by_force,false},
 {load_node,'foo_rel@REDACTED'},
 {load_order,0},
 {load_reason,initial},
 {local_content,false},
 {local_tables,[schema,
                some_table,   <==== This is the table mnesia says doesn't exist
                another_table,
                yet_another_table|...]},
 {master_nodes,[]},
 {memory,3943},
 {ram_copies,['bar_rel@REDACTED','bar_rel@REDACTED']},
 {record_name,schema},
 {record_validation,{schema,3,set}},
 {type,set},
 {size,30},
 {snmp,[]},
 {storage_type,disc_copies},
 {subscribers,...},
 {...}|...]


Every time I try to delete the table, mnesia says it doesn't exist:

(foo_rel@REDACTED)10> mnesia:delete_table(some_table).
{aborted,{no_exists,some_table}}


And I get the following in the log (each time I run the above command):

Oct 17 15:30:50 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction {tid,33075,<0.146.0>} calling #Fun<mnesia_schema.11.13572368> with [] failed: 
Oct 17 15:30:50 somebox.somedomain foo1: [ID 702911 local6.debug]  {aborted,{no_exists,some_table}}

Yet I am able to retrieve info on the table:

(foo_rel@REDACTED)11> mnesia:table_info(some_table,all).
[{access_mode,read_write},
 {active_replicas,[]},
 {arity,4},
 {attributes,[job,key,val]},
 {checkpoints,[]},
 {commit_work,[]},
 {cookie,{{1187,638478,963126},'foo_rel@REDACTED'}},
 {cstruct,{cstruct,some_table,
                   bag,
                   [],
                   ['foo_rel@REDACTED','foo_rel@REDACTED'],
                   [],
                   1,
                   read_write,
                   [],
                   [],
                   false,
                   some_table,
                   [app_job,key,val],
                   [],
                   [],
                   {{1187,638478,...},'foo_rel@REDACTED'},
                   {{3,...},{...}}}},
 {disc_copies,['foo_rel@REDACTED','foo_rel@REDACTED']},
 {disc_only_copies,[]},
 {frag_properties,[]},
 {index,[]},
 {load_by_force,false},
 {load_node,unknown},
 {load_order,1},
 {load_reason,unknown},
 {local_content,false},
 {master_nodes,[]},
 {memory,undefined},
 {ram_copies,[]},
 {record_name,some_table},
 {record_validation,{some_table,4,bag}},
 {type,bag},
 {size,undefined},
 {snmp,[]},
 {storage_type,disc_copies},
 {subscribers,[]},
 {user_properties,...},
 {...}|...]

===

I see that load_node is unknown--probably due to the fact that the other disk
replica is not active (since someotherbox.somedomain is still halted, and for
purposes of this test I cannot bring it back until I restore service to the
primary node).

Again, I have not yet tried to restart with:

  -mnesia max_wait_for_decision Timeout

I figured I would gather as much info as possible, then save what may be the
best (and/or last :-) option left.

Thanks again for all your help,

-Rick


> Clearly the deletion of some_table has really upset mnesia, because I keep
> seeing the following repeated over and over:
> 
> Oct 17 14:56:51 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 31404ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:57:23 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 31412ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:57:54 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32007ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:58:26 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32441ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:58:59 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 32886ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}
> Oct 17 14:59:07 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Transaction log dump initiated by time_threshold: already_dumped
> Oct 17 15:00:05 somebox.somedomain foo1: [ID 702911 local6.debug] Mnesia('foo_rel@REDACTED'): Restarting transaction {tid,33070,<0.95.0>}: in 33459ms {cyclic,'bar_rel@REDACTED',{schema,'______WHOLETABLE_____'},write,write,{tid,1,<4257.71.0>}}


> > RP> Date: Tue, 16 Oct 2007 23:05:36 -0500
> > RP> From: Rick Pettit <rpettit@REDACTED>
> > RP> To: erlang-questions@REDACTED
> > RP> Subject: [erlang-questions] why might mnesia:start() hang?
> > RP> 
> > RP> I seem to have encountered a situation in which I am
> > RP> unable to start mnesia.  Attempts to start mnesia (via
> > RP> mnesia:start/0) hang the erlang shell.
> > RP> 
> > RP> In the scenario below there are 2 physical servers,
> > RP> each running an instance of the foo_rel and
> > RP> bar_rel. The second physical server,
> > RP> someother.somedomain, has been halted prior to starting
> > RP> the nodes somebox.somedomain.
> > RP> 
> > RP> The foo_rel instances contain disc_copy tables--bar_rel
> > RP> instances contain ram_copies only.
> > RP> 
> > RP> (foo_rel@REDACTED)1> application:which_applications().
> > RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> > RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> > RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> > RP> 
> > RP> 
> > RP>    NOTE: there are other applications in this release which *should* be running
> > RP>          but are not, almost certainly due to the fact that mnesia is refusing
> > RP>          to start
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)2> mnesia:info().
> > RP> ===> System info in version "4.3.5", debug level = none <===
> > RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> > RP> use fallback at restart = false
> > RP> running db nodes   = ['foo_rel@REDACTED']
> > RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> > RP> ok
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)3> mnesia:stop().
> > RP> stopped
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)4> mnesia:start().
> > RP> ...shell hangs forever...
> > RP> 
> > RP> 
> > RP> Shell back into the node, try again:
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)1> mnesia:info().
> > RP> ===> System info in version "4.3.5", debug level = none <===
> > RP> opt_disc. Directory "/u1/otp/db/foo_rel" is used.
> > RP> use fallback at restart = false
> > RP> running db nodes   = ['foo_rel@REDACTED']
> > RP> stopped db nodes   = ['foo_rel@REDACTED','bar_rel@REDACTED','bar_rel@REDACTED'] 
> > RP> ok
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)2> application:which_applications().
> > RP> [{sasl,"SASL  CXC 138 11","2.1.5.1"},
> > RP>  {stdlib,"ERTS  CXC 138 10","1.14.5"},
> > RP>  {kernel,"ERTS  CXC 138 10","2.11.5"}]
> > RP> 
> > RP> 
> > RP> (foo_rel@REDACTED)3> mnesia:start().
> > RP> ...hangs forever...
> > RP> 
> > RP> 
> > RP> ======
> > RP> 
> > RP> I cannot seem to figure out:
> > RP> 
> > RP>   1) why mnesia refuses to start
> > RP> 
> > RP>   2) why mnesia:start() hangs forever at the shell
> > RP>     (vs. return an error, etc)
> > RP> 
> > RP> Any applications requiring mnesia tables do a
> > RP> mnesia:wait_for_tables/2 on them.
> > RP> 
> > RP> A special process performs a mnesia:force_load_table/1,
> > RP> if necessary (e.g. when wait_for_tables/2 times out).
> > RP> 
> > RP> Unfortunately, this code doesn't get a chance to run if
> > RP> mnesia itself refuses to start (in many previous test
> > RP> runs the releases started--in some cases the default
> > RP> table load algorithm worked just fine, and in other
> > RP> failure scenarios the force_table_load was
> > RP> necessary--but the system always manged to start until
> > RP> now).
> > RP> 
> > RP> Surely I must just be short on coffee (or sleep) or
> > RP> both. Any help would be greatly appreciated.
> > RP> 
> > RP> -Rick
> > RP> _______________________________________________
> > RP> erlang-questions mailing list
> > RP> erlang-questions@REDACTED
> > RP> http://www.erlang.org/mailman/listinfo/erlang-questions


From dking@REDACTED  Wed Oct 17 22:48:18 2007
From: dking@REDACTED (David King)
Date: Wed, 17 Oct 2007 13:48:18 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
Message-ID: <F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>

> Anyhow - I'm simply trying to see if there are other ways besides  
> process dictionary to simulate "current" variables.  Seems like  
> there is not.

Can you be more specific as to what a "current" variable is?


From joelr1@REDACTED  Wed Oct 17 23:11:08 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Wed, 17 Oct 2007 22:11:08 +0100
Subject: [erlang-questions] Parser combinators
Message-ID: <F39D5C3B-A460-48D8-B528-449F9A017062@gmail.com>

Folks,

Any suggestions on how parser combinators can be elegantly and  
cleanly implemented in Erlang?

State needs to be threaded from combinator to combinator and I don't  
see an option apart from either keeping the state in the process  
dictionary or keeping it in a process launched at the beginning of  
the parsing run.

How could the following bits of Haskell without having to deal with  
state returned from each of reschoice, symbol, commaSep, etc.?

	Thanks, Joel

---

varDeclarations = do
   reschoice [ "Variable", "Variables", "Vars", "Var" ]
   symbol ":"
   decs <- commaSep varDeclaration
   return $ VarDecs decs

whileStatement = do
   reserved "While"
   e <- expr
   reserved "Begin"
   xs <- statements
   reserved "End"
   return $ While e (Compound xs)

ident = do
   c <- identStart
   cs <- many (identLetter)
   return (c:cs)

sign = (char '-' >> return (0-1))
        <|> (char '+' >> return 1)
        <|> return 1


--
http://wagerlabs.com







From kip.macy@REDACTED  Wed Oct 17 23:20:47 2007
From: kip.macy@REDACTED (Kip Macy)
Date: Wed, 17 Oct 2007 14:20:47 -0700
Subject: [erlang-questions] seeking more information on kqueue bug in FreeBSD
Message-ID: <b1fa29170710171420p1fe85a78r7cdc5f3f403ac0e5@mail.gmail.com>

It is my understanding that the following kqueue bug still exists:

http://www.erlang.org/pipermail/erlang-questions/2007-April/026082.html

Can the failure be reproduced with a simple regression test? Or does
it only happen under load? I'd like to fix it, but I'm not sure have
enough information from this mail alone.

 -Kip


From dking@REDACTED  Wed Oct 17 23:38:31 2007
From: dking@REDACTED (David King)
Date: Wed, 17 Oct 2007 14:38:31 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
Message-ID: <653E01CD-9826-4DE7-8F38-19D4DE1CD6D7@ketralnis.com>

> In Erlang, the best example I know at this time is self() - which  
> returns the "current" process ID.  i.e., the returned values of self 
> () changes as it's called within different processes.
> I'm trying to see if there are ways to write functions similar to  
> the context dependent effect of self().  I'm not looking for global  
> variables; context dependent functions such as self() is fine.

Ah, I see. The process dictionary (when used in a read-only fashion,  
as you described earlier) is probably what you're looking for, then.  
That's how mnesia keeps track of your transactions in a construct  
like this:

   mnesia:transaction(fun() ->
                          mnesia:write(..)
                        end), ...

While you don't *specify* anything like current_transaction(),  
mnesia:write() implicitly refers to it, and it sets/gets it in the  
process dictionary. You could write a current_transaction() or  
something that would read it back out of the process dictionary. You  
could also design something like

   Transaction=my_db:create_transaction(),

   Data=my_db:read(Transaction,some_query()),
   NewData=transform(Data),
   my_db:write(Transaction,NewData).

   my_db:commit(Transaction)

Then you're passing your "current transaction" around rather than  
carrying it implicitly. I think that's cleaner. But if you wanted it  
to be implicit, you could use the process dictionary for it like  
mnesia does.


> Some of other examples I can think on top of my head are:
>
> In Java, an example is the Thread.currentThread(), which returns  
> the current thread.
>
> OOP uses this or self as an implicit keyword to retrieve the  
> current object.
>
> In PLT Scheme, there is a facility called parameters, that can be  
> bound for a particular block of code to be a particular value  
> (parameters are retrieved like function calls rather than a naked  
> variable).
>
> In Lisp - special variables can also be bound by context.
>
> Basically - one can bind the value by a particular context (I think  
> in Erlang process is the basic unit for a context).  The main use  
> is to reduce the # of parameters that needs to be explicitly passed  
> in, which obviously makes the function impure (although can be more  
> practical).
>
> Let me know if this explains it better, thanks.
> yc
>



From yinso.chen@REDACTED  Wed Oct 17 23:17:32 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 14:17:32 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
Message-ID: <779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>

On 10/17/07, David King <dking@REDACTED> wrote:
>
>
> > Can you be more specific as to what a "current" variable is?


Certainly.

In Erlang, the best example I know at this time is self() - which returns
the "current" process ID.  i.e., the returned values of self() changes as
it's called within different processes.

I'm trying to see if there are ways to write functions similar to the
context dependent effect of self().  I'm not looking for global variables;
context dependent functions such as self() is fine.

Some of other examples I can think on top of my head are:

In Java, an example is the Thread.currentThread(), which returns the current
thread.

OOP uses this or self as an implicit keyword to retrieve the current object.


In PLT Scheme, there is a facility called parameters, that can be bound for
a particular block of code to be a particular value (parameters are
retrieved like function calls rather than a naked variable).

In Lisp - special variables can also be bound by context.

Basically - one can bind the value by a particular context (I think in
Erlang process is the basic unit for a context).  The main use is to reduce
the # of parameters that needs to be explicitly passed in, which obviously
makes the function impure (although can be more practical).

Let me know if this explains it better, thanks.
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/6ca6b0f4/attachment.htm>

From yinso.chen@REDACTED  Wed Oct 17 23:51:27 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 14:51:27 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <653E01CD-9826-4DE7-8F38-19D4DE1CD6D7@ketralnis.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
	<653E01CD-9826-4DE7-8F38-19D4DE1CD6D7@ketralnis.com>
Message-ID: <779bf2730710171451x771d0ca5yc87831c52eb4d03e@mail.gmail.com>

Great - thanks for the confirmation and explanation.

Cheers,
yc

On 10/17/07, David King <dking@REDACTED> wrote:
>
>
> Ah, I see. The process dictionary (when used in a read-only fashion,
> as you described earlier) is probably what you're looking for, then.
> That's how mnesia keeps track of your transactions in a construct
> like this:
>
>    mnesia:transaction(fun() ->
>                           mnesia:write(..)
>                         end), ...
>
> While you don't *specify* anything like current_transaction(),
> mnesia:write() implicitly refers to it, and it sets/gets it in the
> process dictionary. You could write a current_transaction() or
> something that would read it back out of the process dictionary. You
> could also design something like
>
>    Transaction=my_db:create_transaction(),
>
>    Data=my_db:read(Transaction,some_query()),
>    NewData=transform(Data),
>    my_db:write(Transaction,NewData).
>
>    my_db:commit(Transaction)
>
> Then you're passing your "current transaction" around rather than
> carrying it implicitly. I think that's cleaner. But if you wanted it
> to be implicit, you could use the process dictionary for it like
> mnesia does.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/0e505980/attachment.htm>

From rpettit@REDACTED  Wed Oct 17 23:55:10 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Wed, 17 Oct 2007 16:55:10 -0500
Subject: [erlang-questions] why might mnesia:start() hang?
In-Reply-To: <20071017203000.GA16205@vailsys.com>
References: <20071017040536.GB19639@vailsys.com>
	<Pine.LNX.4.64.0710171038190.18789@amrod.du.uab.ericsson.se>
	<20071017195855.GB26511@vailsys.com>
	<20071017203000.GA16205@vailsys.com>
Message-ID: <20071017215510.GB14093@vailsys.com>

On Wed, Oct 17, 2007 at 03:30:00PM -0500, Rick Pettit wrote:
> On Wed, Oct 17, 2007 at 02:58:55PM -0500, Rick Pettit wrote:
> > On Wed, Oct 17, 2007 at 11:32:14AM +0200, Hakan Mattsson wrote:
> > > 
> > > There may be several causes for this to happen:
> > > 
> > > - It may be the case that some other application has
> > >   encountered a deadlock in its startup. This may for
> > >   example occur if that application is invoking functions
> > >   in the 'application' API during its startup. It may also
> > >   occur if a process dies during the application
> > >   startup. Then its supervisor will not restart the
> > >   process until it has started all its children. 
> > > 
> > > - It could also be that it is Mnesia that refuses to
> > >   start. This may happen if the system first crashes
> > >   during the critical phase in transaction commit and one
> > >   of the other nodes does not come up again. Then Mnesia
> > >   will by default wait indefinitely for the other node to
> > >   be available before it finishes its own startup. See
> > >   the documentation about the Mnesia parameter
> > >   max_wait_for_decision for more info. If you set the
> > >   Mnesia debug level to at least 'verbose' (before you
> > >   start Mnesia) you will get a printout when this happens.

[snip]

> > At this point it would be sufficient to get mnesia running at all on the
> > primary node. I've tried erasing all notion of some_table from the schema,
> > but without luck:
> > 
> > (foo_rel@REDACTED)2> mnesia:delete_table(some_table).                 
> > {aborted,{no_exists,some_table}}
> > (foo_rel@REDACTED)1> mnesia:del_table_copy(schema,'foo_rel@REDACTED'). 
> > {aborted,{no_exists,some_table}}
> 
> Here's some additional information that might help (e.g. I see some_table as
> a member of the local_tables list--that's probably not good if mnesia doesn't
> think the table exists anymore :-)

[snip]

Ok, I finally bit the bullet and restarted the primary node:

  foo_rel@REDACTED

with max_wait_for_decision set to ===> 10000    (i.e. 10 seconds)

I waited for the node to startup, watching the logs--this time after 10
seconds mnesia did jump into action and force a bunch of transactions
to complete, at which the node was *almost* completely up (one application,
a distributed application, was not started for some reason).

Though mnesia started, certain requisite tables were not still not loaded.
I have a distributed application which should have started on the primary
and force loaded all requisite tables (after first failing in an attempt
to mnesia wait for same tables)--but for some reason the distributed 
application controller did not even try to start it.

When I finally shelled into the node and explicitly ran the force
load procedure (which the distributed application had always run
automatically in these instances in the past), the remaining applications came
to life (including the distributed application that formerly refused to
start). At this point the node was finally back in service, ready to
handle production work.

Are there any distributed application experts that might know why the
distributed application controller refused to start the distributed
applicaiton on the primary?

In previous test runs in which no schema modifications were hosed at
shutdown, the distributed application always did the "right thing", 
bouncing from primary to secondary node and back again as one or the
other node was halted--sometimes finding all the tables in good shape,
other times timing out in wait_for_tables/2 and triggering a 
force_load_table/1 on each--but this is the first time the application
refused to start at all (forcing me to do its work by hand).

In either event thank you once again for your help--I have managed to
get the primary node back in service without having to restart the
secondary.

Now I just need to get to the bottom of this distributed application
problem so the system recovers from this on its own next time so I
can sleep in... :-)

-Rick

P.S. To be clear the system I'm working on here has previously recovered
     from the following failure scenario on its own (among many others):

     1) start servers A and B (with nodes running on each which replicate
        disc_copy tables, and a distributed application which bounces
        between A and B, preferring to run on A as primary)

     2) halt server B

     3) halt server A

     4) restart server A

        NOTE: nodes on server A start but the default mnesia table
              load algorithm is unable to load the tables, since it
              believes replicas on B might be more recent, but B is
              unreachable

        NOTE: this is where the distributed application enters the 
              picture--it starts, calls wait_for_tables/2 on all
              requisite tables, and on timeout calls force_load_table/1 
              for each table which could not be loaded by the default
              algorithm.

        NOTE: up until recently this always worked, but then I managed
              to hit a scenario in which a schema transaction was
              affected by the halt, such that mnesia itself would not
              start without -mnesia max_wait_for_desicion 10000--with
              this addition mnesia managed to start but the distributed
              application did not, so the tables which required force
              loading were not force loaded and other dependent
              applications eventually gave up in their own wait_for_tables
              leaving the node in a "bad state" :-)

P.P.S. Thanks for putting up with my ramblings as I try to work through
       this on too much coffee and too little sleep :-)


From richardc@REDACTED  Thu Oct 18 00:13:28 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Thu, 18 Oct 2007 00:13:28 +0200
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>	<20071017181937.GA643@little-black-book.motivity.ca>	<779bf2730710171148w6cbafd4x9d38055b4cf5384a@mail.gmail.com>	<20071017190505.GC643@little-black-book.motivity.ca>
	<779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
Message-ID: <47168908.1080305@it.uu.se>

YC wrote:
> Hopefully this follow-up doesn't make me sound like a troll - but I
> can't help but notice that ets and even Mnesia is recommended before
> process dictionary (which is very heavily frowned upon) in this thread,
> and I am not sure if I understand the reason, unless process dictionary
> is technically inferior or broken.  From FP perspective, neither is
> pure, so why one better than another?
> 
> There must be a reason for the level of discouragement about a built-in
> facility.

It is not a bad question, and I'll try to give some sort of answer.

The main point is that if you think that you need to use the process
dictionary, you are most likely designing your program around (process-)
global variables. This means that your code will contain many more
hidden assumptions than when you are passing around all your parameters
explicitly. It will make it harder to understand, maintain, fix bugs
in, and refactor. In contrast, when only the necessary parameters are
being passed around to functions, it becomes much easier to see patterns
emerge in your code - which groups of functions use a particular
parameter, and which do not. This helps you realize important things
about your code while you are developing it. Even when you group a
bunch of parameters together in a "state" record, the simple fact that
the passing around of the state is visible can help you see which
functions actually need it at all, or use only some specific components
of the state.

The process dictionary is, I find, mainly useful for process-global
options and similar parameters which tend to be orthogonal to the main
functionality of the code. For that sort of thing, it is by definition
difficult to predict in which functions you will suddenly require access
to the value, and to handle it in a functional way you'd be forced to
thread an "options" parameter through all of your calls. So, there are
definitely some cases where it is useful. I suppose that the dire
warnings about using it are mainly due to the fact that if you are
trying to teach old programmers to write good Erlang code, you must give
them a hard slap every time they go near the process dictionary, or they
will start to write Fortran code in Erlang instead of learning good
functional programming patterns.

The use of ets or mnesia is a bit different. In both cases, it is more
or less implied that you use large-ish tables with some sort of uniform
structure or database schema; you wouldn't (well, nobody would expect
you to) create an ets table just to use it as a big old set of global
variables with random contents such as [x -> 42, y -> "hello", ...].
Also, with ets tables at least, you can keep your tables anonymous and
pass the handle around between functions in a visible way, much like
you would do with any functional data structure such as dict. And of
course, the interfaces to ets or mnesia are a bit more heavyweight
than get/put, so it is a little more obvious when reading the code
that there is a table access going on at that point.

The bottom line is that yes, if you have some process-global write-once
parameters, that are more or less orthogonal to the main functionality
of the code, then the process dictionary is a good place to put them.
(Just choose your key names well - if you call out to some other code
which has the same idea and uses the same key, it will overwrite your
value.) The reason for all the discouragement is pedagogical; you don't
want people to start using the process dictionary for problem solving
until they have mastered writing in a functional style.

    /Richard


From thomasl_erlang@REDACTED  Wed Oct 17 23:35:31 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Wed, 17 Oct 2007 14:35:31 -0700 (PDT)
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
Message-ID: <718542.9877.qm@web38814.mail.mud.yahoo.com>


--- YC <yinso.chen@REDACTED> wrote:

> Hi Vance -
> 
> Thanks for the explanations for the different
> approaches in Erlang - this is
> great.
> 
> Hopefully this follow-up doesn't make me sound like
> a troll - but I can't
> help but notice that ets and even Mnesia is
> recommended before process
> dictionary (which is very heavily frowned upon) in
> this thread, and I am not
> sure if I understand the reason, unless process
> dictionary is technically
> inferior or broken.  From FP perspective, neither is
> pure, so why one better
> than another?
> 
> There must be a reason for the level of
> discouragement about a built-in
> facility.

I'm not as opposed to using the process dictionary,
and don't really see a big difference in principle vs
other forms of state. That said, I don't use it much
in practice. I find "stateless" functions to be easier
to use, reuse and test and the process dictionary API
to be a bit unwieldy. 

In practice, with the process dictionary you tend to
do put/get on what is basically global variables (for
that process), so you get problems with reentrancy and
must ensure that you're not calling the function
improperly. You also have the problem of making up
good names for your global variables (no name clashes
please). And there is no good way to save and restore
a whole process dictionary, which is what you'd need
occasionally. (Well, I see there are at least some
BIFs for reading the whole thing. So I may not be
keeping up with the latest and greatest in that.)

Here is an example where things go wrong: you want to
write a map and store the function to use in the
process dictionary to save passing around F as a
parameter.

map(F, Xs) -> 
  put(map_f, F), Ys = map_pd(Xs), erase(map_f), Ys.

map_pd([]) -> [];
map_pd([X|Xs]) -> F = get(map_f), [F(X)|map_pd(Xs)].

So, you set the variable map_f, use it and even clean
it up afterwards. Great. The problem happens when you
want to do a nested map on a list of lists:

map(fun(Xs) -> map(fun f/1, Xs) end, Xss).

When you call the inner map, map_f is set to fun f/1,
then you return and it is erased. The function is not
reentrant, and to get it right, you would have to save
and restore the old value yourself. (Ghastly.) Or you
have to be careful about how you and everybody else
uses it.

As performance goes, however, I think the process
dictionary is noticeably faster than ets. (Didn't
someone post a fast implementation of some ACM
challenge algorithm using the process dictionary a
couple of months ago?) And apart from setting global
"read-only" variables, I very occasionally also use it
for global counters etc. The random library does too. 

Perhaps one could summarize the process dictionary as
a technology for "programming in the small".

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From sean.hinde@REDACTED  Thu Oct 18 00:15:48 2007
From: sean.hinde@REDACTED (Sean Hinde)
Date: Wed, 17 Oct 2007 23:15:48 +0100
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171451x771d0ca5yc87831c52eb4d03e@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
	<653E01CD-9826-4DE7-8F38-19D4DE1CD6D7@ketralnis.com>
	<779bf2730710171451x771d0ca5yc87831c52eb4d03e@mail.gmail.com>
Message-ID: <F760CFF3-1C48-4B3F-8911-DA021940FF46@gmail.com>

I would add a note of caution to your new found understanding. Using  
the process dictionary instead of passing parameters in the vast  
majority of normal erlang code can lead to the most incredibly  
strange bugs, and is almost never what you want.

The use as described in mnesia is a neat hack, and as far as I know  
it is the only place it is used in the whole of OTP.

Probably the best place to learn idiomatic erlang is from the sources  
to the rest of the (less complex) applications that come with OTP,  
and a few of the open source applications out there.

Sean


On 17 Oct 2007, at 22:51, YC wrote:

> Great - thanks for the confirmation and explanation.
>
> Cheers,
> yc
>
> On 10/17/07, David King <dking@REDACTED > wrote:
> Ah, I see. The process dictionary (when used in a read-only fashion,
> as you described earlier) is probably what you're looking for, then.
> That's how mnesia keeps track of your transactions in a construct
> like this:
>
>    mnesia:transaction(fun() ->
>                           mnesia:write(..)
>                         end), ...
>
> While you don't *specify* anything like current_transaction(),
> mnesia:write() implicitly refers to it, and it sets/gets it in the
> process dictionary. You could write a current_transaction() or
> something that would read it back out of the process dictionary. You
> could also design something like
>
>    Transaction=my_db:create_transaction(),
>
>    Data=my_db:read(Transaction,some_query()),
>    NewData=transform(Data),
>    my_db:write(Transaction,NewData).
>
>    my_db:commit(Transaction)
>
> Then you're passing your "current transaction" around rather than
> carrying it implicitly. I think that's cleaner. But if you wanted it
> to be implicit, you could use the process dictionary for it like
> mnesia does.
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From joelr1@REDACTED  Thu Oct 18 00:50:17 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Wed, 17 Oct 2007 23:50:17 +0100
Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
	compiler
Message-ID: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>

Folks,

Roberto Saccon and I agreed to collaborate on the ActionScript  
(ECMAScript 4) to Erlang compiler. We will start with the partial  
yecc grammar that Denis Loutrein put together.

The project is open source and hosted at http://code.google.com/p/ 
jserl/.

My personal goal is to deliver a working compiler within a couple of  
months since the compiler is part of a bigger project for me.

We do need a better name than jserl, though, so how about suggesting  
something warm and fuzzy? How about the Panda Project?

	Thanks, Joel

--
http://wagerlabs.com







From psa@REDACTED  Thu Oct 18 00:42:08 2007
From: psa@REDACTED (=?ISO-8859-1?Q?Paulo_S=E9rgio_Almeida?=)
Date: Wed, 17 Oct 2007 23:42:08 +0100
Subject: [erlang-questions] *current* value?
In-Reply-To: <779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com><14	8408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se><779bf2	730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com><20071017181937.GA643@	little-black-book.motivity.ca><779bf2730710171148w6cbafd4x9d38055b4cf5384a@	mail.gmail.com><20071017190505.GC643@little-black-book.motivity.ca>
	<779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
Message-ID: <47168FC0.2020401@di.uminho.pt>

Hi,

YC wrote:

> Hopefully this follow-up doesn't make me sound like a troll - but I 
> can't help but notice that ets and even Mnesia is recommended before 
> process dictionary (which is very heavily frowned upon) in this thread, 
> and I am not sure if I understand the reason, unless process dictionary 
> is technically inferior or broken.  From FP perspective, neither is 
> pure, so why one better than another?
> 
> There must be a reason for the level of discouragement about a built-in 
> facility.

I think people are exagerating a bit on the discouragement. There are 
bad examples, like using it in general purpose libraries or doing 
intermodule sets and gets. But using it in a small module that contains 
the main loop of the process can result in something readable. This is a 
way to simulate objects, with "instance variables" being in the process 
dictionary. But philosophy apart, the sensible thing to do is:

- for small state, pass it around in a parameter in the process loop, 
e.g. in a record;

- for a large number of *SMALL* entries, or when advanced lookup (like 
select) is needed, use ets;

- for a large state with lots of substructure sharing (e.g. a forest 
with lots of common large sub-trees), use the process dictionary. An 
important difference from ets is that no copying is done: we are only 
referencing process memory, while with ets terms are copied to a 
separate memory area and no subterm sharing is preserved, resulting in 
possibly much larger memory requirements.

Regards,
Paulo



From yinso.chen@REDACTED  Thu Oct 18 02:34:39 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 17 Oct 2007 17:34:39 -0700
Subject: [erlang-questions] *current* value?
In-Reply-To: <718542.9877.qm@web38814.mail.mud.yahoo.com>
References: <779bf2730710171234t236dc497ue39f268a52f20d88@mail.gmail.com>
	<718542.9877.qm@web38814.mail.mud.yahoo.com>
Message-ID: <779bf2730710171734v18b0bab0gda8702e6d2dcb4bb@mail.gmail.com>

Thanks Thomas, Paul, Sean, and Richard for explanations.  Appreciate it.

yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071017/94f7489f/attachment.htm>

From gefla@REDACTED  Thu Oct 18 03:16:42 2007
From: gefla@REDACTED (Gerd Flaig)
Date: Thu, 18 Oct 2007 03:16:42 +0200
Subject: [erlang-questions] *current* value?
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
Message-ID: <87d4vdrzlx.fsf@oxygen.pond.sub.org>

YC <yinso.chen@REDACTED> writes:

> Basically - one can bind the value by a particular context (I think in
> Erlang process is the basic unit for a context).  The main use is to reduce
> the # of parameters that needs to be explicitly passed in, which obviously
> makes the function impure (although can be more practical).

to the contrary, the function becomes impure if it has side effects,
like e.g. modifying the process dictionary.

The number of parameters passed for the current state is normally
exactly 1. If the state has structure, most of the time records are
used.

% create record with some default values
-record(state, {current_confusion=none, current_answer=42}).

start() ->
    % initial state record is constructed with defaults
    server_loop(#state{}).

server_loop(State) ->
    receive
        {get_confused, Confusion} ->
            server_loop(State#state{current_confusion=Confusion});
        {set_answer, Answer} ->
            server_loop(State#state{current_answer=Answer})
    end.


HTH

        Goodbyte, Gerd.
-- 
The last thing one knows in constructing a work is what to put first.
                -- Blaise Pascal



From rsaccon@REDACTED  Thu Oct 18 06:34:33 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 18 Oct 2007 01:34:33 -0300
Subject: [erlang-questions] Erlang or OCaml
In-Reply-To: <4713FC65.4010404@cs.caltech.edu>
References: <536569.99564.qm@web51106.mail.re2.yahoo.com>
	<4713FC65.4010404@cs.caltech.edu>
Message-ID: <eb4177b00710172134w180a3da0rafe2182b02c0dd33@mail.gmail.com>

Great thread, highly informative for anyone considering switching to Erlang,
it even inspired me to summarize this thread in an extended blog post:
http://www.rsaccon.com/2007/10/why-erlang.html

On 10/15/07, Michael Vanier <mvanier@REDACTED> wrote:
>
> Learn both.
>
> Mike
>
> Lone Wolf wrote:
> > Hi.
> > Recently, I have been trying to learn a language other than Java.
> > Erlang seems to be ready for the prime time (used in the commercial
> world).
> > On the other side, OCaml seems to be a cool and capable language.
> > I want to learn something useful and fun.
> > Tell what do you think ?
> > (if you are Erlang die-hard, please put this aside ;))
> >
> > */Deep into that darkness peering, long I stood there, wondering,
> > fearing, Doubting, dreaming dreams no mortal ever dreamed before./*
> > */E.A Poe/*
> > *//*
> > *//*
> >
> > ------------------------------------------------------------------------
> > Catch up on fall's hot new shows
> > <
> http://us.rd.yahoo.com/tv/mail/tagline/falltv/evt=47093/*http://tv.yahoo.com/collections/3658
> >  > on Yahoo! TV. Watch previews, get listings, and more!
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
Roberto Saccon
http://rsaccon.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/3cfa652c/attachment.htm>

From als@REDACTED  Thu Oct 18 07:26:49 2007
From: als@REDACTED (Anthony Shipman)
Date: Thu, 18 Oct 2007 15:26:49 +1000
Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
	compiler
In-Reply-To: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
References: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
Message-ID: <200710181526.49336.als@iinet.net.au>

On Thursday 18 October 2007 08:50, Joel Reymont wrote:
> Folks,
>
> Roberto Saccon and I agreed to collaborate on the ActionScript
> (ECMAScript 4) to Erlang compiler. We will start with the partial
> yecc grammar that Denis Loutrein put together.
>
> The project is open source and hosted at http://code.google.com/p/
> jserl/.
>
> My personal goal is to deliver a working compiler within a couple of
> months since the compiler is part of a bigger project for me.
>
> We do need a better name than jserl, though, so how about suggesting
> something warm and fuzzy? How about the Panda Project?

I've thought of using the SpiderMonkey javascript interpreter as a part of an 
Erlang system. The interpreter could be put into a C node and communicate 
with the rest of the system via messages.

This might be less trouble if you just need an extension language.

-- 
Anthony Shipman                    Mamas don't let your babies 
als@REDACTED                   grow up to be outsourced.


From rsaccon@REDACTED  Thu Oct 18 07:55:13 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 18 Oct 2007 02:55:13 -0300
Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
	compiler
In-Reply-To: <200710181526.49336.als@iinet.net.au>
References: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
	<200710181526.49336.als@iinet.net.au>
Message-ID: <eb4177b00710172255o7521cefbxe16ffff141db082b@mail.gmail.com>

yeah, but in my case I really want it to compile so it runs on the Erlang VM
(I have a use case where I need to do AST transformations and another where
I receive a ready-to-compile AST)

On 10/18/07, Anthony Shipman <als@REDACTED> wrote:
>
> On Thursday 18 October 2007 08:50, Joel Reymont wrote:
> > Folks,
> >
> > Roberto Saccon and I agreed to collaborate on the ActionScript
> > (ECMAScript 4) to Erlang compiler. We will start with the partial
> > yecc grammar that Denis Loutrein put together.
> >
> > The project is open source and hosted at http://code.google.com/p/
> > jserl/.
> >
> > My personal goal is to deliver a working compiler within a couple of
> > months since the compiler is part of a bigger project for me.
> >
> > We do need a better name than jserl, though, so how about suggesting
> > something warm and fuzzy? How about the Panda Project?
>
> I've thought of using the SpiderMonkey javascript interpreter as a part of
> an
> Erlang system. The interpreter could be put into a C node and communicate
> with the rest of the system via messages.
>
> This might be less trouble if you just need an extension language.
>
> --
> Anthony Shipman                    Mamas don't let your babies
> als@REDACTED                   grow up to be outsourced.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
Roberto Saccon
http://rsaccon.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/189bcf59/attachment.htm>

From tobbe@REDACTED  Thu Oct 18 09:43:39 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Thu, 18 Oct 2007 09:43:39 +0200
Subject: [erlang-questions] Parser combinators
In-Reply-To: <F39D5C3B-A460-48D8-B528-449F9A017062@gmail.com>
References: <F39D5C3B-A460-48D8-B528-449F9A017062@gmail.com>
Message-ID: <ff72rb$77d$1@ger.gmane.org>

Have a look here:

http://www.engr.uconn.edu/~jeffm/Source/Erlang/

--Tobbe



Joel Reymont wrote:
> Folks,
> 
> Any suggestions on how parser combinators can be elegantly and  
> cleanly implemented in Erlang?
> 
> State needs to be threaded from combinator to combinator and I don't  
> see an option apart from either keeping the state in the process  
> dictionary or keeping it in a process launched at the beginning of  
> the parsing run.
> 
> How could the following bits of Haskell without having to deal with  
> state returned from each of reschoice, symbol, commaSep, etc.?
> 
> 	Thanks, Joel
> 
> ---
> 
> varDeclarations = do
>    reschoice [ "Variable", "Variables", "Vars", "Var" ]
>    symbol ":"
>    decs <- commaSep varDeclaration
>    return $ VarDecs decs
> 
> whileStatement = do
>    reserved "While"
>    e <- expr
>    reserved "Begin"
>    xs <- statements
>    reserved "End"
>    return $ While e (Compound xs)
> 
> ident = do
>    c <- identStart
>    cs <- many (identLetter)
>    return (c:cs)
> 
> sign = (char '-' >> return (0-1))
>         <|> (char '+' >> return 1)
>         <|> return 1
> 
> 
> --
> http://wagerlabs.com
> 
> 
> 
> 
> 



From tobbe@REDACTED  Thu Oct 18 10:31:42 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Thu, 18 Oct 2007 10:31:42 +0200
Subject: [erlang-questions] blue tail ticket tracker
In-Reply-To: <96fd17a20710171159u6a805dc4w4db59fe58a02e040@mail.gmail.com>
References: <96fd17a20710171159u6a805dc4w4db59fe58a02e040@mail.gmail.com>
Message-ID: <ff75le$fb0$1@ger.gmane.org>

kg9020@REDACTED wrote:
> I am unable to get btt to run on ubuntu 4.1.2  linux 2.6.20-16 - generic
> .  Please look at the following error out put in attached file.  All
> guidance will be welcome

Sorry, but the warranty for BTT expired some centuries ago...
But if you try it via CEAN you might get lucky :-)

--Tobbe

> 
>   Thanks
> 
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
>  bin/btt -i -c  ./btt.conf
> Erlang (BEAM) emulator version 5.4.9 [source] [threads:0]
> 
> ip = {10,254,1,161}
> port = 8337
> docroot = "/home/art/btt_local_install/www"
> dbdir = "/home/art/btt_local_install/db"
> maildir = "/home/art/btt_local_install/mail"
> logdir = "/home/art/btt_local_install/logs/btt"
> cmd = false
> cmd_ip = {127,0,0,1}
> cmd_port = 9338
> Eshell V5.4.9  (abort with ^G)
> 1>
> =INFO REPORT==== 24-Nov-2006::09:51:02 ===
> Yaws: Listening to 10.254.1.161:8337 for servers
>  - http://NoName:8337 under /home/art/btt_local_install/www
> 
> =INFO REPORT==== 5-Dec-2006::15:51:13 ===
> Wrap log "/home/art/btt_local_install/logs/btt/NoName.access"
> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
>        (v)ersion (k)ill (D)b-tables (d)istribution
> /home/art
> 
> 



From joelr1@REDACTED  Thu Oct 18 13:28:36 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Thu, 18 Oct 2007 12:28:36 +0100
Subject: [erlang-questions] *current* value?
In-Reply-To: <87d4vdrzlx.fsf@oxygen.pond.sub.org>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
	<87d4vdrzlx.fsf@oxygen.pond.sub.org>
Message-ID: <D0034986-D5C5-4329-8208-27CBE898F817@gmail.com>

The process dictionary could be used to simplify parsing, I think.

You could write this Haskell parser combinator

whileStatement = do
   reserved "While"
   e <- expr
   reserved "Begin"
   xs <- statements
   reserved "End"
   return $ While e (Compound xs)

like this

reserved("while"),
E = expr(),
reserved("begin"),
Xs = statements(),
reserved("end"),
{while, e, {compound, xs}}.

instead of the more verbose

{S1, ok} = reserved(S, "while"),
{S2, E} = expr(S1),
{S3, ok} = reserved(S2, "begin"),
{S4, Xs} = statements(S3),
{S5, ok} = reserved(S4, "end"),
{S5, {while, e, {compound, xs}}}.

This assumes that parsers throw exceptions upon failure.

What do you think?

--
http://wagerlabs.com







From bjarne@REDACTED  Thu Oct 18 14:38:26 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Thu, 18 Oct 2007 14:38:26 +0200
Subject: [erlang-questions] Erlang workshop 2007
Message-ID: <003b01c81184$30563350$ab1669d4@Dell>

Dear Erlang friends

All the slides from the talks at the 6th ACM Sigplan Erlang Workshop in 
Freiburg on October 5 plus some other material are now available at
http://www.erlang.se/workshop/2007/

The papers are available at the ACM Digital Library.

Best wishes

Bjarne D?cker

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/a7167841/attachment.htm>

From bjarne@REDACTED  Thu Oct 18 14:40:41 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Thu, 18 Oct 2007 14:40:41 +0200
Subject: [erlang-questions] Welcome to EUC 2007
Message-ID: <003c01c81184$30b59160$ab1669d4@Dell>

Dear Erlang friends

Welcome to the Erlang/OTP User Conference 2007. A provisional programme is available at

http://www.erlang.se/euc/07/

Please register soon since the lecture hall is filling up rapidly.

Best regards

Bjarne D?cker
EUC chairman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/4069221d/attachment.htm>

From ulf.wiger@REDACTED  Thu Oct 18 14:48:47 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Thu, 18 Oct 2007 14:48:47 +0200
Subject: [erlang-questions] Microsoft developer vision - functional
	programming
Message-ID: <4717562F.5050809@ericsson.com>


S. Somasegar, Corporate Vice President at Microsoft,
in charge of "Developer Vision", blogs about
functional programming and specifically F#:

http://blogs.msdn.com/somasegar/archive/2007/10/17/f-a-functional-programming-language.aspx

"...ideas from functional languages are helping us address some of the 
biggest challenges facing the industry today, from the impedance 
mismatch between data and objects to the challenges of the multi-core 
and parallel computing space."


BR,
Ulf W


From kg9020@REDACTED  Thu Oct 18 14:57:36 2007
From: kg9020@REDACTED (kg9020@REDACTED)
Date: Thu, 18 Oct 2007 07:57:36 -0500
Subject: [erlang-questions] Fwd: blue tail ticket tracker
In-Reply-To: <96fd17a20710171159u6a805dc4w4db59fe58a02e040@mail.gmail.com>
References: <96fd17a20710171159u6a805dc4w4db59fe58a02e040@mail.gmail.com>
Message-ID: <96fd17a20710180557q3fefdaaesa3098707e46ebe86@mail.gmail.com>

Update last file attachment wast sent in error here is the correct debug
output


---------- Forwarded message ----------
From: kg9020@REDACTED <kg9020@REDACTED>
Date: Oct 17, 2007 1:59 PM
Subject: blue tail ticket tracker
To: erlang-questions <erlang-questions@REDACTED>

I am unable to get btt to run on ubuntu 4.1.2  linux 2.6.20-16 - generic .
Please look at the following error out put in attached file.  All guidance
will be welcome

  Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/4b83a48a/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: btt-script.txt
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/4b83a48a/attachment.txt>

From thomasl_erlang@REDACTED  Thu Oct 18 14:06:54 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Thu, 18 Oct 2007 05:06:54 -0700 (PDT)
Subject: [erlang-questions] *current* value?
In-Reply-To: <D0034986-D5C5-4329-8208-27CBE898F817@gmail.com>
Message-ID: <289864.76426.qm@web38809.mail.mud.yahoo.com>


--- Joel Reymont <joelr1@REDACTED> wrote:

> The process dictionary could be used to simplify
> parsing, I think.
> 
> You could write this Haskell parser combinator
> 
> whileStatement = do
>    reserved "While"
>    e <- expr
>    reserved "Begin"
>    xs <- statements
>    reserved "End"
>    return $ While e (Compound xs)
> 
> like this
> 
> reserved("while"),
> E = expr(),
> reserved("begin"),
> Xs = statements(),
> reserved("end"),
> {while, e, {compound, xs}}.
> 
> instead of the more verbose
> 
> {S1, ok} = reserved(S, "while"),
> {S2, E} = expr(S1),
> {S3, ok} = reserved(S2, "begin"),
> {S4, Xs} = statements(S3),
> {S5, ok} = reserved(S4, "end"),
> {S5, {while, e, {compound, xs}}}.
> 
> This assumes that parsers throw exceptions upon
> failure.
> 
> What do you think?

That notation looks a lot like Prolog's DCG:s (which
used failure and backtracking instead of exceptions).
DCG:s were implemented by the equivalent of a parse
transform, however.

The implementation depends on the grammar, I guess.
For instance, if it needs backtracking, you will also
have to push tokens back onto the input stream when
you throw exceptions to try another grammar rule. But
if the grammar is LL(1) (um, if I remember the jargon
correctly ...) you can parse the input by just
repeatedly looking at the next symbol and can then
safely forget that symbol (or store it, etc). 

I'd probably prefer a parse transform, because it's
cleaner and probably faster, but in a pinch, using the
process dictionary might work. But you are basically
mixing a notational convenience with a method of
implementation.

(Then there's good old yecc, of course.)

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From tim@REDACTED  Thu Oct 18 15:32:13 2007
From: tim@REDACTED (Tim Bates)
Date: Thu, 18 Oct 2007 23:02:13 +0930
Subject: [erlang-questions] *current* value?
In-Reply-To: <289864.76426.qm@web38809.mail.mud.yahoo.com>
References: <289864.76426.qm@web38809.mail.mud.yahoo.com>
Message-ID: <4717605D.9040304@bates.id.au>

Joel Reymont <joelr1@REDACTED> wrote:
> {S1, ok} = reserved(S, "while"),
> {S2, E} = expr(S1),
> {S3, ok} = reserved(S2, "begin"),
> {S4, Xs} = statements(S3),
> {S5, ok} = reserved(S4, "end"),
> {S5, {while, e, {compound, xs}}}.


Thomas Lindgren wrote:
> I'd probably prefer a parse transform, because it's
> cleaner and probably faster, ...


Does anyone have a parse transform for this kind of thing? I currently 
use macros in a pattern like this:


-define(E(Expr), fun(State) -> Expr, State end).
-define(U(Expr), fun(State) -> Expr, NewState end).
-define(EXPRSEQ(Init, List), lists:foldl(fun(Fun, State) -> Fun(State) 
end, Init, List)).


Final = ?EXPRSEQ(Initial, [
   ?U( {ok, NewState} = some_function(State) ),
   ?E( some_other_function(State) ),
   ?U( {ok, NewState} = a_third_function(State) )
])


I use it alot in tests where there's lots of sequential code, and it 
allows me to insert and reorder the expressions without having to update 
all the variable numbers (S1, S2, etc) but it's ugly and involves 
constructing lots of funs. I'd love a parse transform to do something 
like this.

If nobody's got one, I'm prepared to have a crack at writing one if 
anyone has suggestions as to how it might work.

Thanks,
Tim.

-- 
Tim Bates
tim@REDACTED


From kg9020@REDACTED  Thu Oct 18 17:47:50 2007
From: kg9020@REDACTED (kg9020@REDACTED)
Date: Thu, 18 Oct 2007 10:47:50 -0500
Subject: [erlang-questions] blue tail ticket tracker
Message-ID: <96fd17a20710180847u44ed00b5o8bcf0140a0e5e186@mail.gmail.com>

I have found a way to work out the error thanks to Joe's book I have a
direction and a a tool webtool to track down the problem..  Again thanks
...webtool crashdump is great .....

art


On 10/18/07, erlang-questions-request@REDACTED <
erlang-questions-request@REDACTED> wrote:
>
> Send erlang-questions mailing list submissions to
>        erlang-questions@REDACTED
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://www.erlang.org/mailman/listinfo/erlang-questions
> or, via email, send a message with subject or body 'help' to
>        erlang-questions-request <erlang-questions-request@REDACTED>
> @erlang.org <erlang-questions-request@REDACTED>
>
> You can reach the person managing the list at
>        erlang-questions-owner@REDACTED <erlang-questions-owner@REDACTED>
> .org <erlang-questions-owner@REDACTED>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of erlang-questions digest..."
>
> Today's Topics:
>
>   1. ANN: ActionScript (ECMAScript 4) to Erlang        compiler (Joel
> Reymont)
>   2. Re: *current* value? (Paulo S?rgio Almeida)
>   3. Re: *current* value? (YC)
>   4. Re: *current* value? (Gerd Flaig)
>   5. Re: Erlang or OCaml (Roberto Saccon)
>   6. Re: ANN: ActionScript (ECMAScript 4) to Erlang    compiler
>      (Anthony Shipman)
>   7. Re: ANN: ActionScript (ECMAScript 4) to Erlang    compiler
>      (Roberto Saccon)
>   8. Re: Parser combinators (Torbjorn Tornkvist)
>   9. Re: blue tail ticket tracker (Torbjorn Tornkvist)
>
>
> ---------- Forwarded message ----------
> From: Joel Reymont <joelr1@REDACTED>
> To: erlang-questions Questions <erlang-questions@REDACTED>
> Date: Wed, 17 Oct 2007 23:50:17 +0100
> Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
> compiler
> Folks,
>
> Roberto Saccon and I agreed to collaborate on the ActionScript
> (ECMAScript 4) to Erlang compiler. We will start with the partial
> yecc grammar that Denis Loutrein put together.
>
> The project is open source and hosted at http://code.google.com/p/
> jserl/.
>
> My personal goal is to deliver a working compiler within a couple of
> months since the compiler is part of a bigger project for me.
>
> We do need a better name than jserl, though, so how about suggesting
> something warm and fuzzy? How about the Panda Project?
>
>        Thanks, Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Paulo S?rgio Almeida <psa@REDACTED>
> To: YC <yinso.chen@REDACTED>
> Date: Wed, 17 Oct 2007 23:42:08 +0100
> Subject: Re: [erlang-questions] *current* value?
> Hi,
>
> YC wrote:
>
> > Hopefully this follow-up doesn't make me sound like a troll - but I
> > can't help but notice that ets and even Mnesia is recommended before
> > process dictionary (which is very heavily frowned upon) in this thread,
> > and I am not sure if I understand the reason, unless process dictionary
> > is technically inferior or broken.  From FP perspective, neither is
> > pure, so why one better than another?
> >
> > There must be a reason for the level of discouragement about a built-in
> > facility.
>
> I think people are exagerating a bit on the discouragement. There are
> bad examples, like using it in general purpose libraries or doing
> intermodule sets and gets. But using it in a small module that contains
> the main loop of the process can result in something readable. This is a
> way to simulate objects, with "instance variables" being in the process
> dictionary. But philosophy apart, the sensible thing to do is:
>
> - for small state, pass it around in a parameter in the process loop,
> e.g. in a record;
>
> - for a large number of *SMALL* entries, or when advanced lookup (like
> select) is needed, use ets;
>
> - for a large state with lots of substructure sharing (e.g. a forest
> with lots of common large sub-trees), use the process dictionary. An
> important difference from ets is that no copying is done: we are only
> referencing process memory, while with ets terms are copied to a
> separate memory area and no subterm sharing is preserved, resulting in
> possibly much larger memory requirements.
>
> Regards,
> Paulo
>
>
>
>
> ---------- Forwarded message ----------
> From: YC <yinso.chen@REDACTED>
> To: "Thomas Lindgren" <thomasl_erlang@REDACTED>, "Paulo S?rgio Almeida" <
> psa@REDACTED>, "Sean Hinde" <sean.hinde@REDACTED>, "Richard Carlsson"
> <richardc@REDACTED>
> Date: Wed, 17 Oct 2007 17:34:39 -0700
> Subject: Re: [erlang-questions] *current* value?
> Thanks Thomas, Paul, Sean, and Richard for explanations.  Appreciate it.
>
> yc
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Gerd Flaig <gefla@REDACTED>
> To: erlang-questions@REDACTED
> Date: Thu, 18 Oct 2007 03:16:42 +0200
> Subject: Re: [erlang-questions] *current* value?
> YC <yinso.chen@REDACTED> writes:
>
> > Basically - one can bind the value by a particular context (I think in
> > Erlang process is the basic unit for a context).  The main use is to
> reduce
> > the # of parameters that needs to be explicitly passed in, which
> obviously
> > makes the function impure (although can be more practical).
>
> to the contrary, the function becomes impure if it has side effects,
> like e.g. modifying the process dictionary.
>
> The number of parameters passed for the current state is normally
> exactly 1. If the state has structure, most of the time records are
> used.
>
> % create record with some default values
> -record(state, {current_confusion=none, current_answer=42}).
>
> start() ->
>    % initial state record is constructed with defaults
>    server_loop(#state{}).
>
> server_loop(State) ->
>    receive
>        {get_confused, Confusion} ->
>            server_loop(State#state{current_confusion=Confusion});
>        {set_answer, Answer} ->
>            server_loop(State#state{current_answer=Answer})
>    end.
>
>
> HTH
>
>        Goodbyte, Gerd.
> --
> The last thing one knows in constructing a work is what to put first.
>                -- Blaise Pascal
>
>
>
>
> ---------- Forwarded message ----------
> From: "Roberto Saccon" <rsaccon@REDACTED>
> To: "Michael Vanier" <mvanier@REDACTED>
> Date: Thu, 18 Oct 2007 01:34:33 -0300
> Subject: Re: [erlang-questions] Erlang or OCaml
> Great thread, highly informative for anyone considering switching to
> Erlang, it even inspired me to summarize this thread in an extended blog
> post:
> http://www.rsaccon.com/2007/10<http://www.rsaccon.com/2007/10/why-erlang.html>/why-erlang.html
> <http://www.rsaccon.com/2007/10/why-erlang.html>
>
> On 10/15/07, Michael Vanier <mvanier@REDACTED> wrote:
> >
> > Learn both.
> >
> > Mike
> >
> > Lone Wolf wrote:
> > > Hi.
> > > Recently, I have been trying to learn a language other than Java.
> > > Erlang seems to be ready for the prime time (used in the commercial
> > world).
> > > On the other side, OCaml seems to be a cool and capable language.
> > > I want to learn something useful and fun.
> > > Tell what do you think ?
> > > (if you are Erlang die-hard, please put this aside ;))
> > >
> > > */Deep into that darkness peering, long I stood there, wondering,
> > > fearing, Doubting, dreaming dreams no mortal ever dreamed before./*
> > > */E.A Poe/*
> > > *//*
> > > *//*
> > >
> > > ------------------------------------------------------------------------
> >
> > > Catch up on fall's hot new shows
> > > <http://us.rd.yahoo.com/tv/mail<http://us.rd.yahoo.com/tv/mail/tagline/falltv/evt=47093/*http://tv.yahoo.com/collections/3658>
> > /tagline/falltv/evt=47093/*http://tv.yahoo.com/collections/3658
> > <http://us.rd.yahoo.com/tv/mail/tagline/falltv/evt=47093/*http://tv.yahoo.com/collections/3658>
> > >  > on Yahoo! TV. Watch previews, get listings, and more!
> > >
> > >
> > > ------------------------------------------------------------
> > ------------
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman<http://www.erlang.org/mailman/listinfo/erlang-questions>/listinfo/erlang-questions
> > <http://www.erlang.org/mailman/listinfo/erlang-questions>
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman<http://www.erlang.org/mailman/listinfo/erlang-questions>
> > /listinfo/erlang-questions<http://www.erlang.org/mailman/listinfo/erlang-questions>
> >
>
>
>
> --
> Roberto Saccon
> http://rsaccon.com
>
> ---------- Forwarded message ----------
> From: Anthony Shipman <als@REDACTED>
> To: erlang-questions@REDACTED
> Date: Thu, 18 Oct 2007 15:26:49 +1000
> Subject: Re: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
> compiler
> On Thursday 18 October 2007 08:50, Joel Reymont wrote:
> > Folks,
> >
> > Roberto Saccon and I agreed to collaborate on the ActionScript
> > (ECMAScript 4) to Erlang compiler. We will start with the partial
> > yecc grammar that Denis Loutrein put together.
> >
> > The project is open source and hosted at http://code.google.com/p/
> > jserl/.
> >
> > My personal goal is to deliver a working compiler within a couple of
> > months since the compiler is part of a bigger project for me.
> >
> > We do need a better name than jserl, though, so how about suggesting
> > something warm and fuzzy? How about the Panda Project?
>
> I've thought of using the SpiderMonkey javascript interpreter as a part of
> an
> Erlang system. The interpreter could be put into a C node and communicate
> with the rest of the system via messages.
>
> This might be less trouble if you just need an extension language.
>
> --
> Anthony Shipman                    Mamas don't let your babies
> als@REDACTED                   grow up to be outsourced.
>
>
>
> ---------- Forwarded message ----------
> From: "Roberto Saccon" <rsaccon@REDACTED>
> To: "Anthony Shipman" <als@REDACTED>
> Date: Thu, 18 Oct 2007 02:55:13 -0300
> Subject: Re: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
> compiler
> yeah, but in my case I really want it to compile so it runs on the Erlang
> VM (I have a use case where I need to do AST transformations and another
> where I receive a ready-to-compile AST)
>
> On 10/18/07, Anthony Shipman <als@REDACTED> wrote:
> >
> > On Thursday 18 October 2007 08:50, Joel Reymont wrote:
> > > Folks,
> > >
> > > Roberto Saccon and I agreed to collaborate on the ActionScript
> > > (ECMAScript 4) to Erlang compiler. We will start with the partial
> > > yecc grammar that Denis Loutrein put together.
> > >
> > > The project is open source and hosted at http://code.google.com/p/
> > > jserl/.
> > >
> > > My personal goal is to deliver a working compiler within a couple of
> > > months since the compiler is part of a bigger project for me.
> > >
> > > We do need a better name than jserl, though, so how about suggesting
> > > something warm and fuzzy? How about the Panda Project?
> >
> > I've thought of using the SpiderMonkey javascript interpreter as a part
> > of an
> > Erlang system. The interpreter could be put into a C node and
> > communicate
> > with the rest of the system via messages.
> >
> > This might be less trouble if you just need an extension language.
> >
> > --
> > Anthony Shipman                    Mamas don't let your babies
> > als@REDACTED                   grow up to be outsourced.
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman<http://www.erlang.org/mailman/listinfo/erlang-questions>/listinfo/erlang-questions
> > <http://www.erlang.org/mailman/listinfo/erlang-questions>
> >
>
>
>
> --
> Roberto Saccon
> http://rsaccon.com
>
> ---------- Forwarded message ----------
> From: Torbjorn Tornkvist <tobbe@REDACTED>
> To: erlang-questions@REDACTED
> Date: Thu, 18 Oct 2007 09:43:39 +0200
> Subject: Re: [erlang-questions] Parser combinators
> Have a look here:
>
> http://www.engr.uconn.edu/<http://www.engr.uconn.edu/~jeffm/Source/Erlang/>
> ~jeffm/Source/Erlang/ <http://www.engr.uconn.edu/~jeffm/Source/Erlang/>
>
> --Tobbe
>
>
>
> Joel Reymont wrote:
> > Folks,
> >
> > Any suggestions on how parser combinators can be elegantly and
> > cleanly implemented in Erlang?
> >
> > State needs to be threaded from combinator to combinator and I don't
> > see an option apart from either keeping the state in the process
> > dictionary or keeping it in a process launched at the beginning of
> > the parsing run.
> >
> > How could the following bits of Haskell without having to deal with
> > state returned from each of reschoice, symbol, commaSep, etc.?
> >
> >       Thanks, Joel
> >
> > ---
> >
> > varDeclarations = do
> >    reschoice [ "Variable", "Variables", "Vars", "Var" ]
> >    symbol ":"
> >    decs <- commaSep varDeclaration
> >    return $ VarDecs decs
> >
> > whileStatement = do
> >    reserved "While"
> >    e <- expr
> >    reserved "Begin"
> >    xs <- statements
> >    reserved "End"
> >    return $ While e (Compound xs)
> >
> > ident = do
> >    c <- identStart
> >    cs <- many (identLetter)
> >    return (c:cs)
> >
> > sign = (char '-' >> return (0-1))
> >         <|> (char '+' >> return 1)
> >         <|> return 1
> >
> >
> > --
> > http://wagerlabs.com
> >
> >
> >
> >
> >
>
>
>
>
> ---------- Forwarded message ----------
> From: Torbjorn Tornkvist <tobbe@REDACTED>
> To: erlang-questions@REDACTED
> Date: Thu, 18 Oct 2007 10:31:42 +0200
> Subject: Re: [erlang-questions] blue tail ticket tracker
> kg9020@REDACTED wrote:
> > I am unable to get btt to run on ubuntu 4.1.2  linux 2.6.20-16 - generic
> > .  Please look at the following error out put in attached file.  All
> > guidance will be welcome
>
> Sorry, but the warranty for BTT expired some centuries ago...
> But if you try it via CEAN you might get lucky :-)
>
> --Tobbe
>
> >
> >   Thanks
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> >  bin/btt -i -c  ./btt.conf
> > Erlang (BEAM) emulator version 5.4.9 [source] [threads:0]
> >
> > ip = {10,254,1,161}
> > port = 8337
> > docroot = "/home/art/btt_local_install/www"
> > dbdir = "/home/art/btt_local_install/db"
> > maildir = "/home/art/btt_local_install/mail"
> > logdir = "/home/art/btt_local_install/logs/btt"
> > cmd = false
> > cmd_ip = {127,0,0,1}
> > cmd_port = 9338
> > Eshell V5.4.9  (abort with ^G)
> > 1>
> > =INFO REPORT==== 24-Nov-2006::09:51:02 ===
> > Yaws: Listening to 10.254.1.161:8337 for servers
> >  - http://NoName:8337 under /home/art/btt_local_install/www
> >
> > =INFO REPORT==== 5-Dec-2006::15:51:13 ===
> > Wrap log "/home/art/btt_local_install/logs/btt/NoName.access"
> > BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
> >        (v)ersion (k)ill (D)b-tables (d)istribution
> > /home/art
> >
> >
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman<http://www.erlang.org/mailman/listinfo/erlang-questions>
> /listinfo/erlang-questions<http://www.erlang.org/mailman/listinfo/erlang-questions>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071018/67d74b38/attachment.htm>

From kenneth.lundin@REDACTED  Thu Oct 18 17:45:56 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Thu, 18 Oct 2007 17:45:56 +0200
Subject: [erlang-questions] Simple,
	how to do "paging" with mnesia records
In-Reply-To: <20071017175623.M86870@www.botspiritcompany.com>
References: <20071017175623.M86870@www.botspiritcompany.com>
Message-ID: <de422a40710180845u592fc43wf45787ddf9ec711b@mail.gmail.com>

Look at the documentation for QLC
http://www.erlang.org/doc/man/qlc.html

With QLC you can do access mnesia with a cursor in which you specify
how many answers you want.

By the way mnemosyne that you use in your example is a deprecated application
that will be removed from the distribution in the next release i.e
before end of 2007.

/Kenneth (Erlang/OTP team at Ericsson)

On 10/17/07, bbrown <bbrown@REDACTED> wrote:
> For example, in java I might do the following to set at some "start" index
> and then only return X number of records.  This may operate independent of
> the ids in question.
>
> query.setFirstResult(page * pageSize);
> query.setMaxResults(pageSize);
> List data = query.list();
>
>
> What is a mnesia approach for doing this.
>
> Eg, I was going to use something along the following And then maybe do a less
> than link_id > than.  But I havent made it yet.  And yes, I probably need to
> do a little bit more research.  But I thought I would ask before I waste of
> day of work.
>
> page_links() ->
>    Q = query [E || E <- mnesia:table(entity_links),
>                    E <- [#entity_links{link_id=1},
>                          #entity_links{link_id=2}
>                         ]]
>        end,
>    F = fun() -> mnemosyne:eval(Q) end,
>    case mnesia:transaction(F) of
>        {atomic, Result}  -> {ok, Result};
>        {aborted, Reason} -> {error, Reason}
>    end.
>
>
>
> --
> Berlin Brown
> [berlin dot brown at gmail dot com]
> http://botspiritcompany.com/botlist/?
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From sten@REDACTED  Thu Oct 18 19:18:06 2007
From: sten@REDACTED (Sten Kvamme)
Date: Thu, 18 Oct 2007 19:18:06 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
Message-ID: <61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>

Vlad pointed in the right direction, so it's working now (thanks  
Vlad!). But it still puzzles me that I have to mirror the 8 bytes to  
have erl accepting them as a float. In C I just do like this:        
fread(Xrealp, sizeof(Xreal), 1, fpdxf);

Here's the erlang code snippet:

ac_double(Thefile) ->
	<<X:8/binary, B/binary>> = Thefile,
    <<X1,X2,X3,X4,X5,X6,X7,X8>> = X,
    <<Y/float>> = <<X8,X7,X6,X5,X4,X3,X2,X1>>,
	{B,Y}.

Here is the whole thing (remember I have just bought the book ;-)
http://www.kvamme.se/pub/dxf.erl.txt

And here's a small binary dxf file to run on it.
http://www.kvamme.se/pub/b.dxf

I'm learning fast and it's real fun. I can truly recommend the book.

/Sten Kvamme

On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:

> Hi and welcome,
>
> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>> I'm reading a binary file with lots of double data types. Have
>> someone already written some code to convert the double 8 bytes,
>> <<184,162,71,68,75,65,104,64>> to a float?
>
> Do you mean
>    <<X/float>> = <<184,162,71,68,75,65,104,64>>
> ?
>
> regards,
> Vlad



From sten@REDACTED  Thu Oct 18 20:05:45 2007
From: sten@REDACTED (Sten Kvamme)
Date: Thu, 18 Oct 2007 20:05:45 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <de422a40710181048h58fa5ca9ka721781d78a8b389@mail.gmail.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
	<de422a40710181048h58fa5ca9ka721781d78a8b389@mail.gmail.com>
Message-ID: <9AA03841-054B-48D8-8988-8461A7344D60@kvamme.se>

Hello Kenneth,

If I use your example I get a badmatch:

{"init terminating in do_boot",{{badmatch,<<80278 bytes>>}, 
[{dxf,ac_double,1},{dxf,find_entities1,1},{dxf,read_dxf,1}, 
{init,start_it,1},{init,start_em,1}]}}

Strangely enough it seems that the byte order for a float in erlang  
is reverse to the byte order for a float in C, OR there is something  
here that I do not understand yet :-)


/Sten Kvamme

On Oct 18, 2007, at 19:48 , Kenneth Lundin wrote:

> Hi Sten,
>
> I don't understand why you have the code as shown in your example  
> when you can
> do it as simple as this:
>
> ac_double(Thefile) ->
>         <<Y/float, B/binary>> = Thefile,
>         {B,Y}.
>
>
> /Kenneth Erlang/OTP team at Ericsson
>
> On 10/18/07, Sten Kvamme <sten@REDACTED> wrote:
>> Vlad pointed in the right direction, so it's working now (thanks
>> Vlad!). But it still puzzles me that I have to mirror the 8 bytes to
>> have erl accepting them as a float. In C I just do like this:
>> fread(Xrealp, sizeof(Xreal), 1, fpdxf);
>>
>> Here's the erlang code snippet:
>>
>> ac_double(Thefile) ->
>>        <<X:8/binary, B/binary>> = Thefile,
>>    <<X1,X2,X3,X4,X5,X6,X7,X8>> = X,
>>    <<Y/float>> = <<X8,X7,X6,X5,X4,X3,X2,X1>>,
>>        {B,Y}.
>>
>> Here is the whole thing (remember I have just bought the book ;-)
>> http://www.kvamme.se/pub/dxf.erl.txt
>>
>> And here's a small binary dxf file to run on it.
>> http://www.kvamme.se/pub/b.dxf
>>
>> I'm learning fast and it's real fun. I can truly recommend the book.
>>
>> /Sten Kvamme
>>
>> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>>
>>> Hi and welcome,
>>>
>>> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>>>> I'm reading a binary file with lots of double data types. Have
>>>> someone already written some code to convert the double 8 bytes,
>>>> <<184,162,71,68,75,65,104,64>> to a float?
>>>
>>> Do you mean
>>>    <<X/float>> = <<184,162,71,68,75,65,104,64>>
>>> ?
>>>
>>> regards,
>>> Vlad
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>



From tony@REDACTED  Thu Oct 18 20:08:48 2007
From: tony@REDACTED (Tony Rogvall)
Date: Thu, 18 Oct 2007 20:08:48 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
Message-ID: <5475530A-D79E-4019-AE89-17EC69578DF2@rogvall.se>

Erlang defaults to big endian (historical reasons) numbers that is  
high order bytes first,
for intel like machines they use low order bytes first.

use  X/little-float

/Tony

On 18 okt 2007, at 19.18, Sten Kvamme wrote:

> Vlad pointed in the right direction, so it's working now (thanks
> Vlad!). But it still puzzles me that I have to mirror the 8 bytes to
> have erl accepting them as a float. In C I just do like this:
> fread(Xrealp, sizeof(Xreal), 1, fpdxf);
>
> Here's the erlang code snippet:
>
> ac_double(Thefile) ->
> 	<<X:8/binary, B/binary>> = Thefile,
>     <<X1,X2,X3,X4,X5,X6,X7,X8>> = X,
>     <<Y/float>> = <<X8,X7,X6,X5,X4,X3,X2,X1>>,
> 	{B,Y}.
>
> Here is the whole thing (remember I have just bought the book ;-)
> http://www.kvamme.se/pub/dxf.erl.txt
>
> And here's a small binary dxf file to run on it.
> http://www.kvamme.se/pub/b.dxf
>
> I'm learning fast and it's real fun. I can truly recommend the book.
>
> /Sten Kvamme
>
> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>
>> Hi and welcome,
>>
>> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>>> I'm reading a binary file with lots of double data types. Have
>>> someone already written some code to convert the double 8 bytes,
>>> <<184,162,71,68,75,65,104,64>> to a float?
>>
>> Do you mean
>>    <<X/float>> = <<184,162,71,68,75,65,104,64>>
>> ?
>>
>> regards,
>> Vlad
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From sten@REDACTED  Thu Oct 18 20:13:48 2007
From: sten@REDACTED (Sten Kvamme)
Date: Thu, 18 Oct 2007 20:13:48 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <5475530A-D79E-4019-AE89-17EC69578DF2@rogvall.se>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
	<5475530A-D79E-4019-AE89-17EC69578DF2@rogvall.se>
Message-ID: <E3F0247A-3B05-40F1-9E27-7A3C0A14DEEA@kvamme.se>

Thanks, that made it!

/Sten

On Oct 18, 2007, at 20:08 , Tony Rogvall wrote:

> Erlang defaults to big endian (historical reasons) numbers that is  
> high order bytes first,
> for intel like machines they use low order bytes first.
>
> use  X/little-float
>
> /Tony
>
> On 18 okt 2007, at 19.18, Sten Kvamme wrote:
>
>> Vlad pointed in the right direction, so it's working now (thanks
>> Vlad!). But it still puzzles me that I have to mirror the 8 bytes to
>> have erl accepting them as a float. In C I just do like this:
>> fread(Xrealp, sizeof(Xreal), 1, fpdxf);
>>
>> Here's the erlang code snippet:
>>
>> ac_double(Thefile) ->
>> 	<<X:8/binary, B/binary>> = Thefile,
>>     <<X1,X2,X3,X4,X5,X6,X7,X8>> = X,
>>     <<Y/float>> = <<X8,X7,X6,X5,X4,X3,X2,X1>>,
>> 	{B,Y}.
>>
>> Here is the whole thing (remember I have just bought the book ;-)
>> http://www.kvamme.se/pub/dxf.erl.txt
>>
>> And here's a small binary dxf file to run on it.
>> http://www.kvamme.se/pub/b.dxf
>>
>> I'm learning fast and it's real fun. I can truly recommend the book.
>>
>> /Sten Kvamme
>>
>> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>>
>>> Hi and welcome,
>>>
>>> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>>>> I'm reading a binary file with lots of double data types. Have
>>>> someone already written some code to convert the double 8 bytes,
>>>> <<184,162,71,68,75,65,104,64>> to a float?
>>>
>>> Do you mean
>>>    <<X/float>> = <<184,162,71,68,75,65,104,64>>
>>> ?
>>>
>>> regards,
>>> Vlad
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions



From emiller@REDACTED  Thu Oct 18 20:31:39 2007
From: emiller@REDACTED (Evan Miller)
Date: Thu, 18 Oct 2007 11:31:39 -0700
Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
	compiler
In-Reply-To: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
References: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
Message-ID: <ff88r9$nv3$1@ger.gmane.org>

> We do need a better name than jserl, though, so how about suggesting  
> something warm and fuzzy? How about the Panda Project?
> 

To continue the primate theme of the various ECMAScript compilers 
(IronMonkey, ScreamingMonkey, SpiderMonkey, etc.), how about "Lemur"? 
 From Wikipedia:

"The Pygmy Mouse Lemur is the smallest of the mouse lemurs and the 
smallest primate in the world, only weighing around 30 grams... Its 
small size and nocturnal nature made it difficult to locate for over a 
century, and was rediscovered in the Kirindy forest in western 
Madagascar in 1993."

http://en.wikipedia.org/wiki/Pygmy_Mouse_Lemur

Evan



From kenneth.lundin@REDACTED  Thu Oct 18 19:48:46 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Thu, 18 Oct 2007 19:48:46 +0200
Subject: [erlang-questions] double to float
In-Reply-To: <61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<61167A5A-9078-42FE-8F9D-F54A9CE687E9@kvamme.se>
Message-ID: <de422a40710181048h58fa5ca9ka721781d78a8b389@mail.gmail.com>

Hi Sten,

I don't understand why you have the code as shown in your example when you can
do it as simple as this:

ac_double(Thefile) ->
        <<Y/float, B/binary>> = Thefile,
        {B,Y}.


/Kenneth Erlang/OTP team at Ericsson

On 10/18/07, Sten Kvamme <sten@REDACTED> wrote:
> Vlad pointed in the right direction, so it's working now (thanks
> Vlad!). But it still puzzles me that I have to mirror the 8 bytes to
> have erl accepting them as a float. In C I just do like this:
> fread(Xrealp, sizeof(Xreal), 1, fpdxf);
>
> Here's the erlang code snippet:
>
> ac_double(Thefile) ->
>        <<X:8/binary, B/binary>> = Thefile,
>    <<X1,X2,X3,X4,X5,X6,X7,X8>> = X,
>    <<Y/float>> = <<X8,X7,X6,X5,X4,X3,X2,X1>>,
>        {B,Y}.
>
> Here is the whole thing (remember I have just bought the book ;-)
> http://www.kvamme.se/pub/dxf.erl.txt
>
> And here's a small binary dxf file to run on it.
> http://www.kvamme.se/pub/b.dxf
>
> I'm learning fast and it's real fun. I can truly recommend the book.
>
> /Sten Kvamme
>
> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>
> > Hi and welcome,
> >
> > On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
> >> I'm reading a binary file with lots of double data types. Have
> >> someone already written some code to convert the double 8 bytes,
> >> <<184,162,71,68,75,65,104,64>> to a float?
> >
> > Do you mean
> >    <<X/float>> = <<184,162,71,68,75,65,104,64>>
> > ?
> >
> > regards,
> > Vlad
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From per@REDACTED  Fri Oct 19 01:19:32 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 19 Oct 2007 01:19:32 +0200 (CEST)
Subject: [erlang-questions] double to float
In-Reply-To: <5475530A-D79E-4019-AE89-17EC69578DF2@rogvall.se>
Message-ID: <200710182319.l9INJWsJ040302@pluto.hedeland.org>

Tony Rogvall <tony@REDACTED> wrote:
>
>Erlang defaults to big endian (historical reasons) numbers that is  
>high order bytes first,

...in binaries. Yes but... - the important thing is that they *have* a
default endianness, and so binaries are "portable" unless you mess them
up. Whereas the file (written and) read with (fwrite() and) fread() from
C in the way Sten described cannot be fread() on a differently-endian
machine.

>for intel like machines they use low order bytes first.
>
>use  X/little-float

So, to emulate the non-portable C way, I guess that should be
X/native-float.

--Per Hedeland


From joelr1@REDACTED  Fri Oct 19 02:00:13 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Fri, 19 Oct 2007 01:00:13 +0100
Subject: [erlang-questions] ANN: ActionScript (ECMAScript 4) to Erlang
	compiler
In-Reply-To: <ff88r9$nv3$1@ger.gmane.org>
References: <81FA9CC1-4A90-4A8C-AF91-3271AA42835B@gmail.com>
	<ff88r9$nv3$1@ger.gmane.org>
Message-ID: <0D36625E-CCC0-4A59-9092-E87805D09C75@gmail.com>


On Oct 18, 2007, at 7:31 PM, Evan Miller wrote:

>> To continue the primate theme of the various ECMAScript compilers
> (IronMonkey, ScreamingMonkey, SpiderMonkey, etc.), how about "Lemur"?

I like it!!!

My only wish is that the Erlang syntax was more declarative.

	Thanks, Joel

--
http://wagerlabs.com







From jeffm@REDACTED  Fri Oct 19 06:14:43 2007
From: jeffm@REDACTED (jm)
Date: Fri, 19 Oct 2007 14:14:43 +1000
Subject: [erlang-questions] Using select in a Port
Message-ID: <47182F33.2070600@ghostgun.com>

In writing a simple port I've hit a problem with using select(). Select 
is used so that the program can read from both stdin and another file 
descriptor. The program runs fine while receiving commands on stdin. It 
continues to run fine when the second file descriptor opens. It only 
fails after something arrives on the second file descriptor is processed 
and the results sent to the erlang node in then loops printing the debug 
output, to stderr,

read_cmd return(-1)
buf empty. retval = 1
note_read
          note_read end while for fd = 3
note_read return
read_cmd return(-1)
buf empty. retval = 1
note_read
          note_read end while for fd = 3

and so on.


What I think is the relevent bits of code is below. It appears that 
select() is returning saying that something is waiting to be read on 
stdin, read_cmd() then fails to read the two byte header and returns -1. 
Simply put what's the fix? Have I missed something to do with select()? 
The read_cmd() and read_exact() are take from "Programming Erlang" and 
http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs
and should be correct.

Any help appreciated.

Jeff.


/**************** code extract ***************/
int main() {
.....
  maxfd = setup_select(nlist, &readfds);

   while ((retval = select(maxfd + 1, &readfds, NULL, NULL, NULL)) >= 0) {
     if FD_ISSET(0, &readfds) {
       memset(buf, 0, BUFFER_SIZE);
       read_cmd(buf, &size);
       index = 0;

       if (*buf != '\0') {
         if (ei_decode_version(buf, &index, &version) ||
             ei_decode_tuple_header(buf, &index, &arity) ||
             ei_decode_atom(buf, &index, command)) {
           fprintf(stderr, "ei_decode failure...\r\n");
           fprintf(stderr, "buf=(%p) %s\r\n", buf, buf);
           fprintf(stderr, "index=%d\r\n", index);
           exit(4);
         }
         process_command(command, buf, &index, nlist);
       } else {
         fprintf(stderr, "buf empty. retval = %d\r\n", retval);
       }
     }
     fprintf(stderr, "note_read\n");
     /* check other file descriptors */
     note_read(nlist, &readfds);

     maxfd = setup_select(nlist, &readfds);
   }
.....
}

/***************** setup_select *********************/
int setup_select(note_t *nlist, fd_set *readfds) {
   int topfd, maxfd = 0;

   /* pselect initialisation */
   FD_ZERO(readfds);
   FD_SET(0, readfds);

   topfd = note_setup_select(nlist, readfds);
   if (topfd > maxfd)
     maxfd = topfd;

   return maxfd;
}

/***************** read_cmd *************/
int read_cmd(byte *buf, int *size)
{
   int len;

   if (read_exact(buf, 2) != 2) {
     fprintf(stderr, "read_cmd return(-1)\r\n");
     return(-1);
   }
   len = (buf[0] << 8) | buf[1];

   fprintf(stderr, "read_cmd len = %d\r\n", len);
   if (len > *size) {
     buf = (byte *) realloc(buf, len);
     if (buf == NULL)
       return -1;
     *size = len;
   }
   return read_exact(buf, len);
}

/************* read_exact ***************/
int read_exact(byte *buf, int len) {
   int i, got = 0;

   do {
     if ((i = read(0, buf+got, len-got)) <= 0)
       return(i);
     got += i;
   } while (got<len);

   return(len);
}


From per@REDACTED  Fri Oct 19 08:36:52 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 19 Oct 2007 08:36:52 +0200 (CEST)
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <47182F33.2070600@ghostgun.com>
Message-ID: <200710190636.l9J6aqrS049403@pluto.hedeland.org>

jm <jeffm@REDACTED> wrote:
>
>What I think is the relevent bits of code is below. It appears that 
>select() is returning saying that something is waiting to be read on 
>stdin,

select() (or poll()) never says that - it says "If you try to read()
from (e.g.) stdin, it won't block even if the file descriptor is in
blocking mode".

>int read_cmd(byte *buf, int *size)
>{
>   int len;
>
>   if (read_exact(buf, 2) != 2) {
>     fprintf(stderr, "read_cmd return(-1)\r\n");
>     return(-1);
>   }

This is plain broken - the return value must be checked properly:

   > 0 but less than you wanted: No error, just that all the data hasn't
   arrived yet - save what you got and try again for the rest the next
   time select/poll triggers.

   0: end-of-file - not an error (the other end of a socket or pipe was
   closed, or someone hit ^D on a tty), but you don't want to read from
   this descriptor ever again - select/poll will keep reporting the fd
   as "ready".

   < 0: Error - depending on the context of your program you may need to
   check for errno EINTR or EAGAIN, which could be non-fatal in which
   case you just ignore the whole thing. Otherwise fatal, and the same
   processing as for EOF typically applies. (This is pretty rare and
   most likely a bug in your program, e.g. trying to read from a closed
   descriptor).

Of course your problem could be something else too...

--Per Hedeland


From grimlog@REDACTED  Fri Oct 19 10:48:44 2007
From: grimlog@REDACTED (Michael Schreckenbauer)
Date: Fri, 19 Oct 2007 10:48:44 +0200
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <47182F33.2070600@ghostgun.com>
References: <47182F33.2070600@ghostgun.com>
Message-ID: <200710191048.48625.grimlog@gmx.de>

Hi,

Am Freitag, 19. Oktober 2007 schrieb jm:
> What I think is the relevent bits of code is below. It appears that
> select() is returning saying that something is waiting to be read on
> stdin, read_cmd() then fails to read the two byte header and returns -1.
> Simply put what's the fix? Have I missed something to do with select()?

imo you missed something to do with read ;)

> The read_cmd() and read_exact() are take from "Programming Erlang" and
> http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_progr
>ams and should be correct.

They are, for reading from stdin. See below.

> Any help appreciated.
> Jeff.
>
>
> /**************** code extract ***************/
> int main() {
> .....
>   maxfd = setup_select(nlist, &readfds);
>
>    while ((retval = select(maxfd + 1, &readfds, NULL, NULL, NULL)) >= 0) {

This proceeds, if at least one(!) of the fds you gave won't block, when you do 
a read on it. That could be stdin or any other.

>      if FD_ISSET(0, &readfds) {
>        memset(buf, 0, BUFFER_SIZE);
>        read_cmd(buf, &size);

Here you check the right fd, but read_cmd always uses stdin.
The nonblocking fd is 3, your file, as you can see in the debug output. So 
read_cmd must fail. It needs an additional parameter holding the fd you want 
to use.

Regards,
Michael

-- 
I don't want to bore you, but there's nobody else around for me to bore.


From per@REDACTED  Fri Oct 19 15:18:47 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 19 Oct 2007 15:18:47 +0200 (CEST)
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710191048.48625.grimlog@gmx.de>
Message-ID: <200710191318.l9JDIlXu057260@pluto.hedeland.org>

Michael Schreckenbauer <grimlog@REDACTED> wrote:
>
>This proceeds, if at least one(!) of the fds you gave won't block, when you do 
>a read on it. That could be stdin or any other.
>
>>      if FD_ISSET(0, &readfds) {
>>        memset(buf, 0, BUFFER_SIZE);
>>        read_cmd(buf, &size);
>
>Here you check the right fd, but read_cmd always uses stdin.
>The nonblocking fd is 3, your file, as you can see in the debug output. So 
>read_cmd must fail. It needs an additional parameter holding the fd you want 
>to use.

No, this is code is correct per se - if fd 0 is ready, he calls a
function that reads only from fd 0, which is clearly OK. The other fd's
are handled in note_read(), which isn't included in the mail but is
where the debug output is produced. Regardless of that, select says that
fd 0 *is* "ready" - the error is in assuming that this means that there
is (some specific amount of) data to read.

--Per Hedeland


From grimlog@REDACTED  Fri Oct 19 15:45:17 2007
From: grimlog@REDACTED (Michael Schreckenbauer)
Date: Fri, 19 Oct 2007 15:45:17 +0200
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710191318.l9JDIlXu057260@pluto.hedeland.org>
References: <200710191318.l9JDIlXu057260@pluto.hedeland.org>
Message-ID: <200710191545.23114.grimlog@gmx.de>

Hi Per,

Am Freitag, 19. Oktober 2007 schrieb Per Hedeland:
> Michael Schreckenbauer <grimlog@REDACTED> wrote:
> >This proceeds, if at least one(!) of the fds you gave won't block, when
> > you do a read on it. That could be stdin or any other.
> >
> >>      if FD_ISSET(0, &readfds) {
> >>        memset(buf, 0, BUFFER_SIZE);
> >>        read_cmd(buf, &size);
> >
> >Here you check the right fd, but read_cmd always uses stdin.
> >The nonblocking fd is 3, your file, as you can see in the debug output. So
> >read_cmd must fail. It needs an additional parameter holding the fd you
> > want to use.
>
> No, this is code is correct per se - if fd 0 is ready, he calls a
> function that reads only from fd 0, which is clearly OK. The other fd's
> are handled in note_read(), which isn't included in the mail but is
> where the debug output is produced. Regardless of that, select says that
> fd 0 *is* "ready" - the error is in assuming that this means that there
> is (some specific amount of) data to read.

yes, of course you're right. Must have been to early for me ;)

> --Per Hedeland

Thanks,
Michael


From exta7@REDACTED  Fri Oct 19 18:22:38 2007
From: exta7@REDACTED (Zvi)
Date: Fri, 19 Oct 2007 09:22:38 -0700 (PDT)
Subject: [erlang-questions] Binary comprehension (was Re: double to
 float)
In-Reply-To: <13286617.post@talk.nabble.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>
	<f49fd40d0710161341s361d682fw74d5496ca5e19850@mail.gmail.com>
	<13286617.post@talk.nabble.com>
Message-ID: <13286772.post@talk.nabble.com>


Found the answer:
---------------------------------------
bc.erl file:
--------------------------------------
-module(bc).
-compile([binary_comprehension]).
-export([test/0]).

test() -> 
	Bin = <<"hello">>,
	[B || <<B:8>> <= Bin].
---------------------------------------
1> c(bc).        
{ok,bc}
2> bc:test().
"hello"

 OR:

---------------------------------------
bc.erl file:
--------------------------------------
-module(bc).
-export([test/0]).

test() -> 
	Bin = <<"hello">>,
	[B || <<B:8>> <= Bin].
---------------------------------------
1> c(bc,[binary_comprehension]).
{ok,bc}
2> bc:test().                   
"hello"
---------------------------------------

Still doesn't know, how to run this from shell.
Also, too bad we can't write something like this:

  <<B:8 || B<-[1,2,3]>>.

output should be:

  <<1,2,3>>

Zvi



Zvi wrote:
> 
> 1> Bin = <<"hello">>.
> <<hello>>
> 2> [B || <<B:8>> <= Bin].
> ** 1: binary generator in code not compiled with the binary comprehension
> flag *
> 
> How do I enable binary comprehension?
> Do I really need to recompile entire Erlang/OTP with this?
> Or does it referes to the compile directive in the .erl source, something
> like this?
> 
> -compile([binary_comprehension]).
> 
> or
> 
> c(mod.erl, [binary_comprehension]).
> 
> If yes, then how do I use it from the Erlang shell?
> 
> TIA,
> Zvi
> 
> 
> 
> 
> 
> 
> 
> 
> Matthew O'Gorman wrote:
>> 
>> this works great if you compile with binary_comprehension
>> 
>> List_of_floats = [Float || << Float/float >> <= Binary].
>> 
>> mog
>> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>>> Yes thanks, but I have the file in a binary variable and would like to
>>> do
>>> something like this:
>>>
>>> {<<F/float>>,B} = split_binary(Thefile, 8),
>>>
>>> or maybe
>>>
>>> <<F:8/binary,B/binary>> = Thefile,
>>> <<X/float>> =  F,
>>>
>>>
>>> /Sten
>>>
>>>
>>> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>>>
>>>
>>> <<X/float>> = <<184,162,71,68,75,65,104,64>>
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/double-to-float-tf4635969.html#a13286772
Sent from the Erlang Questions mailing list archive at Nabble.com.



From exta7@REDACTED  Fri Oct 19 18:22:52 2007
From: exta7@REDACTED (Zvi)
Date: Fri, 19 Oct 2007 09:22:52 -0700 (PDT)
Subject: [erlang-questions] Binary comprehension (was Re: double to float)
In-Reply-To: <f49fd40d0710161341s361d682fw74d5496ca5e19850@mail.gmail.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>
	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>
	<6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>
	<f49fd40d0710161341s361d682fw74d5496ca5e19850@mail.gmail.com>
Message-ID: <13286617.post@talk.nabble.com>


1> Bin = <<"hello">>.
<<hello>>
2> [B || <<B:8>> <= Bin].
** 1: binary generator in code not compiled with the binary comprehension
flag *

How do I enable binary comprehension?
Do I really need to recompile entire Erlang/OTP with this?
Or does it referes to the compile directive in the .erl source, something
like this?

-compile([binary_comprehension]).

or

c(mod.erl, [binary_comprehension]).

If yes, then how do I use it from the Erlang shell?

TIA,
Zvi








Matthew O'Gorman wrote:
> 
> this works great if you compile with binary_comprehension
> 
> List_of_floats = [Float || << Float/float >> <= Binary].
> 
> mog
> On 10/16/07, Sten Kvamme <sten@REDACTED> wrote:
>> Yes thanks, but I have the file in a binary variable and would like to do
>> something like this:
>>
>> {<<F/float>>,B} = split_binary(Thefile, 8),
>>
>> or maybe
>>
>> <<F:8/binary,B/binary>> = Thefile,
>> <<X/float>> =  F,
>>
>>
>> /Sten
>>
>>
>> On Oct 16, 2007, at 20:43 , Vlad Dumitrescu wrote:
>>
>>
>> <<X/float>> = <<184,162,71,68,75,65,104,64>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 

-- 
View this message in context: http://www.nabble.com/double-to-float-tf4635969.html#a13286617
Sent from the Erlang Questions mailing list archive at Nabble.com.



From kostis@REDACTED  Fri Oct 19 19:48:58 2007
From: kostis@REDACTED (Kostis Sagonas)
Date: Fri, 19 Oct 2007 20:48:58 +0300
Subject: [erlang-questions] Binary comprehension (was Re: double to
	float)
In-Reply-To: <13286772.post@talk.nabble.com>
References: <6BC569B9-F1D3-48CA-8FEB-BBC69D58245E@kvamme.se>	<95be1d3b0710161143t6b470683h87578328ad6db102@mail.gmail.com>	<6C8EB2AC-82F5-4841-82D6-4F808CEB7173@kvamme.se>	<f49fd40d0710161341s361d682fw74d5496ca5e19850@mail.gmail.com>	<13286617.post@talk.nabble.com>
	<13286772.post@talk.nabble.com>
Message-ID: <4718EE0A.2010109@cs.ntua.gr>

Zvi wrote:
> Found the answer:
> .... SNIP .....
> 
> Still doesn't know, how to run this from shell.
> Also, too bad we can't write something like this:
> 
>   <<B:8 || B<-[1,2,3]>>.
> 

Full support for binary comprehensions and other goodies like
the ones you want will be available in the upcoming R12.

Stay tuned -- it's coming soon in an OTP system near you.

Kostis


From erlang@REDACTED  Fri Oct 19 21:44:22 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Fri, 19 Oct 2007 21:44:22 +0200
Subject: [erlang-questions] *current* value?
In-Reply-To: <D0034986-D5C5-4329-8208-27CBE898F817@gmail.com>
References: <779bf2730710162203i68e981balb22c5ccab793e243@mail.gmail.com>
	<148408C0A2D44A41AB295D74E1839975036767A3@esealmw105.eemea.ericsson.se>
	<779bf2730710171110x2d572eeoa2ba3980afd83a2f@mail.gmail.com>
	<22B63B9E-9189-4E5E-B1E4-089AE5C49FB4@ketralnis.com>
	<779bf2730710171156u162f0ecbn5a91ff7a43c39e59@mail.gmail.com>
	<F8A22AE8-4CDC-402F-ABA9-5845FAE2F1BD@ketralnis.com>
	<779bf2730710171417k42cb512ga5b6b74653eaf682@mail.gmail.com>
	<87d4vdrzlx.fsf@oxygen.pond.sub.org>
	<D0034986-D5C5-4329-8208-27CBE898F817@gmail.com>
Message-ID: <9b08084c0710191244j6b4b2bbcn7ae9f32249fc2a18@mail.gmail.com>

What do I think?

I think if we do this we'll have to spend several years writing
thousands of lines
of text explaining why monads are really easy to understand.

Given the *enormous* difficutly that people have in understanding monads in
Haskell it might be inadvisable to introduce them into Erlang.

Some amazing command in emacs would be nice

meta-x monadify-region might daisy-chain an additional dummy argument through
a sequence of function calls in the manner you describe :-)

/Joe


On 10/18/07, Joel Reymont <joelr1@REDACTED> wrote:
> The process dictionary could be used to simplify parsing, I think.
>
> You could write this Haskell parser combinator
>
> whileStatement = do
>    reserved "While"
>    e <- expr
>    reserved "Begin"
>    xs <- statements
>    reserved "End"
>    return $ While e (Compound xs)
>
> like this
>
> reserved("while"),
> E = expr(),
> reserved("begin"),
> Xs = statements(),
> reserved("end"),
> {while, e, {compound, xs}}.
>
> instead of the more verbose
>
> {S1, ok} = reserved(S, "while"),
> {S2, E} = expr(S1),
> {S3, ok} = reserved(S2, "begin"),
> {S4, Xs} = statements(S3),
> {S5, ok} = reserved(S4, "end"),
> {S5, {while, e, {compound, xs}}}.
>
> This assumes that parsers throw exceptions upon failure.
>
> What do you think?
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ahmed.nawras@REDACTED  Fri Oct 19 22:16:33 2007
From: ahmed.nawras@REDACTED (Ahmed Ali)
Date: Sat, 20 Oct 2007 00:16:33 +0400
Subject: [erlang-questions] Error running crypto:start() under windows
Message-ID: <d5f75cfa0710191316t2787ccf5n8b9787864dfa5cb9@mail.gmail.com>

Hi,

During playing with erlyweb under windows, when running erlydb:start()
function, I got into troubles. The reason was due to crypto:start() giving
me the following error:

{error,{shutdown,{crypto_app,start,[normal,[]]}}}
=INFO REPORT==== 19-Oct-2007::23:30:11 ===
    application: crypto
    exited: {shutdown,{crypto_app,start,[normal,[]]}}
    type: temporary

I spent sometime trying to figure out the issue but finally I did solve it
and I thought I'd inform others about it in the mailing list. The solution
is to copy "libeay32.dll" to C:\Windows directory. I hope this is helpful.

Best regards,

Ahmed Al-Issaei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071020/0a20effd/attachment.htm>

From hughperkins@REDACTED  Fri Oct 19 22:42:42 2007
From: hughperkins@REDACTED (Hugh Perkins)
Date: Fri, 19 Oct 2007 22:42:42 +0200
Subject: [erlang-questions] Microsoft developer vision - functional
	programming
In-Reply-To: <4717562F.5050809@ericsson.com>
References: <4717562F.5050809@ericsson.com>
Message-ID: <837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>

On 10/18/07, Ulf Wiger (TN/EAB) <ulf.wiger@REDACTED> wrote:
> "...ideas from functional languages

The biggest idea from FP for parallel seems to be daring to copy all
the data to each thread, rather than letting multiple threads read a
single central copy.

The downside of that is obviously that it's insanely inefficient when
there is lots of data involved.

I guess this is already being done, but... presumably one could
optimize the runtime so that the data gets stored in "pages" (or
equivalent), and as long as no thread modifies the data, the "pages"
are shared, but if any thread tries to modify a page, that page
immediately gets forked into an independent copy for that particular
thread.


From exta7@REDACTED  Sat Oct 20 06:04:28 2007
From: exta7@REDACTED (Zvi)
Date: Fri, 19 Oct 2007 21:04:28 -0700 (PDT)
Subject: [erlang-questions]  Erlang VM 32 and 64 bit interoperability
Message-ID: <13297530.post@talk.nabble.com>


Can Erlang node running on 32 bit Linux exchange messages with Erlang node
running on 64 bit Linux?

TIA
Zvi
-- 
View this message in context: http://www.nabble.com/Erlang-VM-32-and-64-bit-interoperability-tf4654187.html#a13297530
Sent from the Erlang Questions mailing list archive at Nabble.com.



From ckerr@REDACTED  Sat Oct 20 06:02:35 2007
From: ckerr@REDACTED (Cameron Kerr)
Date: Sat, 20 Oct 2007 17:02:35 +1300
Subject: [erlang-questions] Microsoft developer vision - functional
	programming
In-Reply-To: <837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
References: <4717562F.5050809@ericsson.com>
	<837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
Message-ID: <20071020040235.GA24943@bruce.local>

On Fri, Oct 19, 2007 at 10:42:42PM +0200, Hugh Perkins wrote:
> On 10/18/07, Ulf Wiger (TN/EAB) <ulf.wiger@REDACTED> wrote:
> > "...ideas from functional languages
> 
> The biggest idea from FP for parallel seems to be daring to copy all
> the data to each thread, rather than letting multiple threads read a
> single central copy.

Please correct me if I'm wrong, but isn't one of the benefits of
Erlang's (in particular) single-assignment restriction that you only
need to pass around a pointer, and never need to copy the data (because
the data cannot be modified once it is first assigned). Obviously, you
would need to copy the data if sending to a different machine.

-- 
Cameron Kerr <ckerr@REDACTED>
Telecommunications Teaching Fellow
University of Otago, New Zealand



From saifi.khan@REDACTED  Sat Oct 20 07:08:25 2007
From: saifi.khan@REDACTED (Saifi Khan)
Date: Sat, 20 Oct 2007 10:38:25 +0530 (IST)
Subject: [erlang-questions] 64-bit erlang
Message-ID: <Pine.LNX.4.64.0710201032001.7841@speed.localdomain>

Hi:

Is it possible to compile a clean 64-bit version of erlang 
from the source tar ball available on erlang.org ?

There is a paper that does talk about porting Beam on AMD64.

I am running a clean 64-bit AMD64 X2 box with  a 
LFS (Linux From Scratch) based Linux kernel 2.6.22 and gcc 4.1.x.

Can some experienced erlangers share their insight on this ?

Thanks in advance.

thanks
Saifi.

-------------------------------------------------------
TWINCLING Society            http://www.twincling.org/
freedom of innovation        Hyderabad AP, India.
-------------------------------------------------------





From saifi.khan@REDACTED  Sat Oct 20 07:17:39 2007
From: saifi.khan@REDACTED (Saifi Khan)
Date: Sat, 20 Oct 2007 10:47:39 +0530 (IST)
Subject: [erlang-questions] Microsoft developer vision - functional
 programming
In-Reply-To: <4717562F.5050809@ericsson.com>
References: <4717562F.5050809@ericsson.com>
Message-ID: <Pine.LNX.4.64.0710201043010.8047@speed.localdomain>

On Thu, 18 Oct 2007, Ulf Wiger (TN/EAB) wrote:

> Date: Thu, 18 Oct 2007 14:48:47 +0200
> From: "Ulf Wiger (TN/EAB)" <ulf.wiger@REDACTED>
> To: EQ <erlang-questions@REDACTED>
> Subject: [erlang-questions] Microsoft developer vision - functional
>     programming
> 
> 
> S. Somasegar, Corporate Vice President at Microsoft,
> in charge of "Developer Vision", blogs about
> functional programming and specifically F#:
> 
> http://blogs.msdn.com/somasegar/archive/2007/10/17/f-a-functional-programming-language.aspx
> 
> "...ideas from functional languages are helping us address some of the 
> biggest challenges facing the industry today, from the impedance 
> mismatch between data and objects to the challenges of the multi-core 
> and parallel computing space."
> 
> 
> BR,
> Ulf W

Vision and Microsoft ? interesting.

COM, VC++, .NET/C blunt were the revolutionary technologies that MS sold
to valued customers, now its harping F# (to cleane'm all).
Your Potential, Our License ! 
Yeah.

thanks
Saifi.


From bjorn@REDACTED  Sat Oct 20 08:00:00 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 20 Oct 2007 08:00:00 +0200
Subject: [erlang-questions] Erlang VM 32 and 64 bit interoperability
In-Reply-To: <13297530.post@talk.nabble.com>
References: <13297530.post@talk.nabble.com>
Message-ID: <o1myuepbq7.fsf@erix.ericsson.se>

Zvi <exta7@REDACTED> writes:

> Can Erlang node running on 32 bit Linux exchange messages with Erlang node
> running on 64 bit Linux?

Yes.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From bjorn@REDACTED  Sat Oct 20 08:03:30 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 20 Oct 2007 08:03:30 +0200
Subject: [erlang-questions] 64-bit erlang
In-Reply-To: <Pine.LNX.4.64.0710201032001.7841@speed.localdomain>
References: <Pine.LNX.4.64.0710201032001.7841@speed.localdomain>
Message-ID: <o1ir52pbkd.fsf@erix.ericsson.se>

Saifi Khan <saifi.khan@REDACTED> writes:

> Hi:
> 
> Is it possible to compile a clean 64-bit version of erlang 
> from the source tar ball available on erlang.org ?

Yes.

If your C compiler produces 64-bit object files by default, it
will happen automatically. By now, the 64-bit emulator should be
as stable and reliable as the 32-bit one.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From jeffm@REDACTED  Sat Oct 20 09:00:13 2007
From: jeffm@REDACTED (jm)
Date: Sat, 20 Oct 2007 17:00:13 +1000
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710190636.l9J6aqrS049403@pluto.hedeland.org>
References: <200710190636.l9J6aqrS049403@pluto.hedeland.org>
Message-ID: <4719A77D.9090608@ghostgun.com>

Changed the read_cmd and the main function based on your advise to make 
it store the current offset and to continue reading where it had left 
off. After starting the port and calling a benign function upto few 
times the read returns 0, ie the debug prints,

module:list().
read_cmd start curpos = 0
read_cmd curpos = 2
read_cmd len = 10 desired=10
read_cmd returned message
list
     note_read
              note_read return
call_port: {ok,[]}
{ok,[]}                      %%%% <-- correct result sent and recieved.
3> read_cmd start curpos = 0
read_cmd returned 0: Success(0)


it appears somthing is causing erlang to close stdin to the port, but 
I'm scratching my head as to why as no error is logged by the node. All 
message sent from my module to the erlang node are sent using the ei 
library and write_cmd as shown in the previous email.

Jeff.

/*
  * inputs
  *    buf     a pointer the buffer to place the message in
  *    size    size of buf
  *    curpos  current offset into buf. Should be set to 0 before
  *            initial call
  *
  * returns  < 0 on read() error
  *         == 0 on read() returning 0
  *            1 when there is more to read
  *            2 when the message is complete
  */
int read_cmd(char *buf, int *size, int *curpos)
{
   int len;
   int count;
   int desired;

   fprintf(stderr, "read_cmd start curpos = %d\r\n", *curpos);
   if (*curpos < 2) {
     /* read header */
     count = read(0, buf + *curpos, 2 - *curpos);

     if (count <= 0)
       return(count); /* Error or fd is closed */

     *curpos += count;
     if (*curpos < 2)
       return(1);
   }
   fprintf(stderr, "read_cmd curpos = %d\r\n", *curpos);
   /* calculate the total message length and
    * the desired amount to read taking into account
    * the ammount already read
    */
   len = (buf[0] << 8) | buf[1];
   desired = len - *curpos + 2;
   fprintf(stderr, "read_cmd len = %d desired=%d\r\n", len, desired);

   /* check buffer size and realloc if necessary */
   if (len > *size) {
     buf = (char *) realloc(buf, len);
     if (buf == NULL)
       return -1;
     *size = len;
   }

   /* read message body */
   count = read(0, buf + *curpos, desired);
   if (count <= 0)
     return(0);

   *curpos += count;
   return(2);
}

/****** main ******/

int main() {
....
  while ((retval = select(maxfd + 1, &readfds, NULL, NULL, NULL)) >= 0) {
     if FD_ISSET(0, &readfds) {
       memset(inbuf, 0, BUFFER_SIZE);
       index = 0;
       result = read_cmd(inbuf, &size, &cmdpos);
       if (result == 0) {
         fprintf(stderr, "read_cmd returned 0: %s(%d)\r\n", 
strerror(errno), errno);
         exit(1);
       } else if (result < 0) {
         fprintf(stderr, "read_cmd returned < 0: %s(%d)\r\n", 
strerror(errno), errno);
         exit(1);
       } else if (result == 1) {
         fprintf(stderr, "read_cmd need to read more");
       } else {
         fprintf(stderr, "read_cmd returned message\r\n");
         /* must add two(2) to inbuf pointer to skip message length 
header */
         if (ei_decode_version(inbuf+2, &index, &version) ||
             ei_decode_tuple_header(inbuf+2, &index, &arity) ||
             ei_decode_atom(inbuf+2, &index, command)) {
           fprintf(stderr, "ei_decode failure...\r\n");
           fprintf(stderr, "buf=(%p) %s\r\n", inbuf, inbuf);
           fprintf(stderr, "index=%d\r\n", index);
           exit(4);
         }
         process_command(command, inbuf+2, &index, nlist);
         /* reset position of inbuf */
         cmdpos = 0;
       }
     }
     fprintf(stderr, "note_read\n");
     /* check other file descriptors */
     note_read(nlist, &readfds);

     maxfd = setup_select(nlist, &readfds);
   }
....
}


Per Hedeland wrote:
> jm <jeffm@REDACTED> wrote:
>> What I think is the relevent bits of code is below. It appears that 
>> select() is returning saying that something is waiting to be read on 
>> stdin,
> 
> select() (or poll()) never says that - it says "If you try to read()
> from (e.g.) stdin, it won't block even if the file descriptor is in
> blocking mode".
> 
>> int read_cmd(byte *buf, int *size)
>> {
>>   int len;
>>
>>   if (read_exact(buf, 2) != 2) {
>>     fprintf(stderr, "read_cmd return(-1)\r\n");
>>     return(-1);
>>   }
> 
> This is plain broken - the return value must be checked properly:
> 
>    > 0 but less than you wanted: No error, just that all the data hasn't
>    arrived yet - save what you got and try again for the rest the next
>    time select/poll triggers.
> 
>    0: end-of-file - not an error (the other end of a socket or pipe was
>    closed, or someone hit ^D on a tty), but you don't want to read from
>    this descriptor ever again - select/poll will keep reporting the fd
>    as "ready".
> 
>    < 0: Error - depending on the context of your program you may need to
>    check for errno EINTR or EAGAIN, which could be non-fatal in which
>    case you just ignore the whole thing. Otherwise fatal, and the same
>    processing as for EOF typically applies. (This is pretty rare and
>    most likely a bug in your program, e.g. trying to read from a closed
>    descriptor).
> 
> Of course your problem could be something else too...
> 
> --Per Hedeland



From tobias.gerdin@REDACTED  Sat Oct 20 11:50:14 2007
From: tobias.gerdin@REDACTED (Tobias Gerdin)
Date: Sat, 20 Oct 2007 11:50:14 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
Message-ID: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>

Hi there,

I do not understand why the single-assignment restriction exists in  
Erlang (even though I very much enjoy functional programming) given  
that processes do not share state. What is the problem with mutable  
state in a concurrent system when other processes cannot access other  
processes' state?

I was hoping someone could enlighten me on this matter.


Regards,

Tobias






From ulf@REDACTED  Sat Oct 20 13:33:46 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 20 Oct 2007 13:33:46 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
Message-ID: <8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>

While single assignment and pattern matching go very well
together, it's a little bit less obvious how mutable state and
pattern matching would combine as nicely. Erlang's current
semantics for binding and matching make it quite straight-
forward to reason about even very complex programming
patterns.

BR,
Ulf W

2007/10/20, Tobias Gerdin <tobias.gerdin@REDACTED>:
> Hi there,
>
> I do not understand why the single-assignment restriction exists in
> Erlang (even though I very much enjoy functional programming) given
> that processes do not share state. What is the problem with mutable
> state in a concurrent system when other processes cannot access other
> processes' state?
>
> I was hoping someone could enlighten me on this matter.
>
>
> Regards,
>
> Tobias
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ckerr@REDACTED  Sat Oct 20 13:37:01 2007
From: ckerr@REDACTED (Cameron Kerr)
Date: Sun, 21 Oct 2007 00:37:01 +1300
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
Message-ID: <4AEF3AC9-359A-410D-B6B8-95452E2865C8@cs.otago.ac.nz>

If you can only assign to a variable once, you cannot have race  
conditions.

Another (possible) benefit of single assignment is that it means you  
only ever need to pass a pointer to the memory. In mutable languages,  
you cannot be sure that the source won't be modified, so you copy the  
memory. With single assignment, it is always safe to pass a pointer,  
knowing that the source cannot change. A corollary of that is that  
message-passing is much faster than it if you would do it in eg. C.  
Of course, this does not hold when sending to a different host, which  
has a different memory space.

I would be interested in knowing if this is what Erlang does in fact do.

On 20/10/2007, at 10:50 PM, Tobias Gerdin wrote:

> Hi there,
>
> I do not understand why the single-assignment restriction exists in
> Erlang (even though I very much enjoy functional programming) given
> that processes do not share state. What is the problem with mutable
> state in a concurrent system when other processes cannot access other
> processes' state?
>
> I was hoping someone could enlighten me on this matter.
>
>
> Regards,
>
> Tobias
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 
? Cameron Kerr  ? ? ckerr@REDACTED ?
? ?
? Telecommunications Teaching Fellow & SysAdmin ?
? ? http://humbledown.org/blog/ ? ? New: 027 7175 244 ?

"Technological advances are not made by sadomasochistic, cultic, tool- 
worshipping pain freaks." ? http://freeshells.ch/~revence/myths.txt




From ckerr@REDACTED  Sat Oct 20 14:01:47 2007
From: ckerr@REDACTED (Cameron Kerr)
Date: Sun, 21 Oct 2007 01:01:47 +1300
Subject: [erlang-questions] Performance of the append/reverse idiom
In-Reply-To: <9b08084c0707160053w3ddcc029wc73da0de6458ab94@mail.gmail.com>
References: <469B10F8.2030005@baus.net>
	<9b08084c0707160053w3ddcc029wc73da0de6458ab94@mail.gmail.com>
Message-ID: <DA8A3C4B-9956-4FAA-B059-A2A06CBC1BAF@cs.otago.ac.nz>

So what is the time complexity of reverse/1? I know that Erlang  
intercepts calls to lists:reverse/1 to make performance a lot better.  
Given the description, I get the impression that it might be O(1),  
and so internally there might be some doubly-linked list sort of  
thing going on.

What is the internal representation of a list such that it makes it  
much cheaper?

PS. Please CC in any responses, I don't regularly follow this list.

On 16/07/2007, at 7:53 PM, Joe Armstrong wrote:

> Is reverse a bottleneck?
>
> Very rarely.
>
> The are some *extremely* rare cases where people write code to handle
> both reversed
> and non-reverse lists (I think the library routines for sort remember
> if the current
> list being sorted is in normal or reversed order) but this is only  
> useful when
> you're interested in making a tiny gain in efficiency (it also make  
> the code
> a  mess to read).
>
> The most common bottleneck are in the I/O system. Parsing inputs (in
> any language)
> is always a slow - other bottleneck are usually in the process
> structure - using a centralized server that serializes everything can
> make nasty bottlenecks.
>
> /Joe
>
>
> On 7/16/07, Christopher Baus <christopher@REDACTED> wrote:
>> Hi,
>>
>> I'm still going through Joe's book, but as a C/C++ hack, I can't help
>> but have some reservations about performance.  One common idiom is to
>> append to the front of a list and then reverse the list when the  
>> append
>> is complete.  I assume this a O(n) operation.  Is it common for  
>> reverse
>> to be a bottleneck in many applications?
>>
>> Thanks,
>>
>> Chris
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 
? Cameron Kerr  ? ? ckerr@REDACTED ?
? ?
? Telecommunications Teaching Fellow & SysAdmin ?
? ? http://humbledown.org/blog/ ? ? New: 027 7175 244 ?

"Technological advances are not made by sadomasochistic, cultic, tool- 
worshipping pain freaks." ? http://freeshells.ch/~revence/myths.txt




From per.gustafsson@REDACTED  Sat Oct 20 14:36:27 2007
From: per.gustafsson@REDACTED (Per Gustafsson)
Date: Sat, 20 Oct 2007 14:36:27 +0200
Subject: [erlang-questions] Performance of the append/reverse idiom
In-Reply-To: <DA8A3C4B-9956-4FAA-B059-A2A06CBC1BAF@cs.otago.ac.nz>
References: <469B10F8.2030005@baus.net>	<9b08084c0707160053w3ddcc029wc73da0de6458ab94@mail.gmail.com>
	<DA8A3C4B-9956-4FAA-B059-A2A06CBC1BAF@cs.otago.ac.nz>
Message-ID: <4719F64B.7040308@it.uu.se>

Cameron Kerr wrote:
> So what is the time complexity of reverse/1? I know that Erlang  
> intercepts calls to lists:reverse/1 to make performance a lot better.  
> Given the description, I get the impression that it might be O(1),  
> and so internally there might be some doubly-linked list sort of  
> thing going on.
>
> What is the internal representation of a list such that it makes it  
> much cheaper?
>
> PS. Please CC in any responses, I don't regularly follow this list.
>
> On 16/07/2007, at 7:53 PM, Joe Armstrong wrote:
>
>   
lists:reverse is O(N), but it is implemented in C rather than in Erlang 
which makes it faster. The internal representation of lists is just 
linked cons cells where each cons cell takes up two words of heap space.

Per



>> Is reverse a bottleneck?
>>
>> Very rarely.
>>
>> The are some *extremely* rare cases where people write code to handle
>> both reversed
>> and non-reverse lists (I think the library routines for sort remember
>> if the current
>> list being sorted is in normal or reversed order) but this is only  
>> useful when
>> you're interested in making a tiny gain in efficiency (it also make  
>> the code
>> a  mess to read).
>>
>> The most common bottleneck are in the I/O system. Parsing inputs (in
>> any language)
>> is always a slow - other bottleneck are usually in the process
>> structure - using a centralized server that serializes everything can
>> make nasty bottlenecks.
>>
>> /Joe
>>
>>
>> On 7/16/07, Christopher Baus <christopher@REDACTED> wrote:
>>     
>>> Hi,
>>>
>>> I'm still going through Joe's book, but as a C/C++ hack, I can't help
>>> but have some reservations about performance.  One common idiom is to
>>> append to the front of a list and then reverse the list when the  
>>> append
>>> is complete.  I assume this a O(n) operation.  Is it common for  
>>> reverse
>>> to be a bottleneck in many applications?
>>>
>>> Thanks,
>>>
>>> Chris
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>
>>>       
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>     
>
>   



From richardc@REDACTED  Sat Oct 20 14:51:03 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Sat, 20 Oct 2007 14:51:03 +0200
Subject: [erlang-questions] Performance of the append/reverse idiom
In-Reply-To: <DA8A3C4B-9956-4FAA-B059-A2A06CBC1BAF@cs.otago.ac.nz>
References: <469B10F8.2030005@baus.net>	<9b08084c0707160053w3ddcc029wc73da0de6458ab94@mail.gmail.com>
	<DA8A3C4B-9956-4FAA-B059-A2A06CBC1BAF@cs.otago.ac.nz>
Message-ID: <4719F9B7.5060006@it.uu.se>

Cameron Kerr wrote:
> So what is the time complexity of reverse/1? I know that Erlang  
> intercepts calls to lists:reverse/1 to make performance a lot better.  
> Given the description, I get the impression that it might be O(1),  
> and so internally there might be some doubly-linked list sort of  
> thing going on.
> 
> What is the internal representation of a list such that it makes it  
> much cheaper?
> 
> PS. Please CC in any responses, I don't regularly follow this list.

There is no magic here. Reverse is an O(n) operation; it's just that
the built-in version (written in C) has a smaller constant factor than
a version implemented in Erlang.

Anyhow, doing a reverse at the end of an operation to get the elements
in the right order is not really a problem, if you consider that since
it is O(n), this just means a fairly small constant cost per element.
(The cost includes the additional memory allocation and higher pressure
on the garbage collector.) You have to consider that the alternative
is to rewrite the algorithm to produce the result in the right order
directly, and (assuming there are no destructive operations), no matter
what you do, this will also have its additional costs per element.

The right thing to do is to stop worrying and learn to love reverse!
It will let you write the main algorithm in a clearer way, which will
typically turn out to be more efficient, even if you have to include a
call to reverse at the end.

    /Richard


From kosik@REDACTED  Sat Oct 20 14:54:13 2007
From: kosik@REDACTED (Matej Kosik)
Date: Sat, 20 Oct 2007 14:54:13 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
	<8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
Message-ID: <4719FA75.3040707@fiit.stuba.sk>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ulf Wiger wrote:
> While single assignment and pattern matching go very well
> together, it's a little bit less obvious how mutable state and
> pattern matching would combine as nicely. Erlang's current
> semantics for binding and matching make it quite straight-
> forward to reason about even very complex programming
> patterns.

Doesn't shadowing of meanings of variables in Erlang similarly complicate reasoning? (in one
position some variable has some meaning and elsewhere in the term suddenly has different meaning).

Non-functional features of Erlang are present in its fundamental constructs:
- - send
- - receive
- - spawn
These cannot be modelled in (purely) functional languages, can they? How?

> 
> BR,
> Ulf W
> 


Regards
- --
Matej Ko??k
ICQ: 300133844
skype: matej_kosik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGfp1L+CaXfJI/hgRAsz6AKDPh8of9p2ZjNcbdepLMGNYqDP+PwCfUwfq
oeKj320tszFHLxPXTG8Fzig=
=vv1n
-----END PGP SIGNATURE-----


From ulf@REDACTED  Sat Oct 20 15:52:23 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sat, 20 Oct 2007 15:52:23 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <4719FA75.3040707@fiit.stuba.sk>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
	<8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
	<4719FA75.3040707@fiit.stuba.sk>
Message-ID: <8209f740710200652l5c4ba58n396f51329d868e76@mail.gmail.com>

2007/10/20, Matej Kosik <kosik@REDACTED>:
>
> Doesn't shadowing of meanings of variables in Erlang similarly
> complicate reasoning? (in one position some variable has some
> meaning and elsewhere in the term suddenly has different meaning).

Well, it could, but in most cases where it would be confusing, the
compiler will give you a warning.

Personally, I try to use some naming convention for variables to
minimize the risk of confusion.


> Non-functional features of Erlang are present in its fundamental constructs:
> - - send
> - - receive
> - - spawn
> These cannot be modelled in (purely) functional languages, can they? How?

Almost by definition, the answer is "no", I think?  (:

(I guess I should let the real language experts answer this, but it's Saturday,
and I don't have anything better to do right now. I trust I'll get my fingers
slapped if I write something stupid).

At least it's difficult. Haskell uses 'monads', to clearly mark where the code
has side-effects. This allows the compiler to trust that all code outside of
the monad is pure. Other languages can use the fact that these operations
are contained within the runtime, and can optimize them. Felix* does this,
for example; I don't know about Concurrent ML. I believe that when Felix
participated in the Language Shootout, its compiler reduced the concurrency
benchmark down to a function that did nothing**. This doesn't mean that
Felix models "fibres" as being pure, but as long as the effect of the
impurity is understood, a smart compiler can sometimes eliminate it,
or at least simplify it significantly. I guess this is sort of a grey area. (:

It is partly connected to the fact that most of the surroundings are pure,
and very well understood. Erlang can, for example, decide to pass a
pointer rather than copy a message, since there are no user-level
operations that can mess with the low-level representation of the
message. There was once a paper suggesting that a gen_server:call()
could be reduced to a function call, under certain circumstances
(http://www.erlang.se/workshop/2002/Stenman.pdf), but it
suggested dynamic checking to ensure that the conditions were met,
since it would be difficult to do statically in Erlang.

(OHaskell not only performs this check statically, it enforces it:
http://www.cs.chalmers.se/~nordland/ohaskell/)

Erlang's approach is fairly pragmatic. It has basically tried to make sure
that all non-pure operations can be modeled using spawn, send and receive,
but the only parts of an erlang program that the compiler can trust to be
free from side-effects are clause heads, the left hand side of an expression
(pattern matching), and guard expressions. Every (non-guard) function call
can potentially result in a side-effect. Some BIFs are also known to be pure
and can be optimized by the compiler.

* http://www.felix-lang.org - a OCaml/C++ hybrid of sorts
** This Felix entry was disqualified, which caused some debate. Google
will tell you more.

BR,
Ulf W


From anders.nygren@REDACTED  Sat Oct 20 16:08:41 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Sat, 20 Oct 2007 09:08:41 -0500
Subject: [erlang-questions] Strange optimization result
Message-ID: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>

Hi
I am playing around with Tim Bray's widefinder problem.
Actually I am playing with Steve V's latest version
see http://steve.vinoski.net/blog/2007/10/18/ok-just-one-more-wf/

In it he is using a dict for Boyer-Moore searching, the dict is just
a map from char code to shift amount i.e. the keys are 1 - 255
and the values are ints.
I figured that it would be faster to just have a tuple with the shift values.
So I changed the code to generate a tuple instead of a dict. And the
lookups changed from
dict:fetch(C1, Tbl)
to
element(C1,Tbl)

To my big surprise the program now runs slower.
On a file with 1M rows approx. 192 M.

Steve's original code, using dics, as reported by time
real    0m20.444s
user    0m37.718s
sys     0m0.612s

My modified, using a tuple
real    0m25.849s
user    0m44.203s
sys     0m3.332s

Interesting to note is the high value for sys in my version.
Also the tuple version does not manage to max out the CPU use
it peaks at ~85%, while the dict version gets 100% CPU load.
The program starts 64 erlang processes on my dual core laptop.

My only idea is that this is caused by some pathological CPU cache
behaviour. Does anyone have a better explanation?

/Anders


From r.kelsall@REDACTED  Sat Oct 20 16:44:49 2007
From: r.kelsall@REDACTED (Richard Kelsall)
Date: Sat, 20 Oct 2007 15:44:49 +0100
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
Message-ID: <471A1461.50207@millstream.com>

Tobias Gerdin wrote:
> I do not understand why the single-assignment restriction exists in  
> Erlang (even though I very much enjoy functional programming) given  
> that processes do not share state. What is the problem with mutable  
> state in a concurrent system when other processes cannot access other  
> processes' state?

Erlang is fundamentally about extreme reliability. The features have
been chosen to make it as reliable as possible and it's remarkably
good at this. Part of this is achieved by giving the programmer
a very clear view of how the code works and when it goes wrong
the ability to easily remove the bug. So a 'variable' is local
to a function and is only 'assigned' a value at one place in the
code. This reduces the complexity of the code. If a function were
to overwrite a variable then crash it would be more difficult to
see what went wrong because we would have deleted the original value
and if the variable were assigned values at several places in the
function we might not know which assignment statement gave it the
wrong value. Single assignment effectively gives each value a different
name to clarify the code.

Consider also the difficulties of hot code change - which increases
reliability by decreasing the cost of fixing bugs - if you keep values
in places other than local to functions in single assignment variables.
Would you interrupt a long running for-loop to change the code, but
try to retain the values of the variables? Complicated. The function
call in Erlang is the transition from the old to the new version of
the code. Loops - which is where a traditional language would normally
use destructive assignment - are modeled in Erlang as a sequence of
tail-recursive function calls passing the loop state across in the
parameters. So the programmer is encouraged to write short functions
and carefully consider the state required next time round the 'loop'.
This makes hot code change easier.

I think that's about right. Corrections welcome.


Richard.



From exta7@REDACTED  Sat Oct 20 17:36:08 2007
From: exta7@REDACTED (Zvi)
Date: Sat, 20 Oct 2007 08:36:08 -0700 (PDT)
Subject: [erlang-questions] Microsoft developer vision - functional
 programming
In-Reply-To: <20071020040235.GA24943@bruce.local>
References: <4717562F.5050809@ericsson.com>
	<837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
	<20071020040235.GA24943@bruce.local>
Message-ID: <13311055.post@talk.nabble.com>


this would work if all the data stored in the global VM heap, but in Erlang
b/c of Soft Real-Time constraint, each process has it's own heap, so you
can't just pass pointer to object from another process' heap, which can be
garbage collected any time.
 But Erlang have some optimizations, for large binaries, which are stored in
the global heap, and using reference counting for garbage collection.

Zvi


Cameron Kerr-2 wrote:
> 
> On Fri, Oct 19, 2007 at 10:42:42PM +0200, Hugh Perkins wrote:
> 
> Please correct me if I'm wrong, but isn't one of the benefits of
> Erlang's (in particular) single-assignment restriction that you only
> need to pass around a pointer, and never need to copy the data (because
> the data cannot be modified once it is first assigned). Obviously, you
> would need to copy the data if sending to a different machine.
> 
> -- 
> Cameron Kerr <ckerr@REDACTED>
> Telecommunications Teaching Fellow
> University of Otago, New Zealand
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 

-- 
View this message in context: http://www.nabble.com/Microsoft-developer-vision---functional-programming-tf4646804.html#a13311055
Sent from the Erlang Questions mailing list archive at Nabble.com.



From exta7@REDACTED  Sat Oct 20 17:39:02 2007
From: exta7@REDACTED (Zvi)
Date: Sat, 20 Oct 2007 08:39:02 -0700 (PDT)
Subject: [erlang-questions] Microsoft developer vision - functional
 programming
In-Reply-To: <20071020040235.GA24943@bruce.local>
References: <4717562F.5050809@ericsson.com>
	<837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
	<20071020040235.GA24943@bruce.local>
Message-ID: <13311055.post@talk.nabble.com>


this would work if all the data stored in the global VM heap, but in Erlang
b/c of Soft Real-Time constraint, each process has it's own heap, so you
can't just pass pointer to object from another process' heap, which can be
garbage collected any time.
 But Erlang have some optimizations, for large binaries, which are stored in
the global heap, and using reference counting for garbage collection.

Zvi


Cameron Kerr-2 wrote:
> 
> On Fri, Oct 19, 2007 at 10:42:42PM +0200, Hugh Perkins wrote:
> 
> Please correct me if I'm wrong, but isn't one of the benefits of
> Erlang's (in particular) single-assignment restriction that you only
> need to pass around a pointer, and never need to copy the data (because
> the data cannot be modified once it is first assigned). Obviously, you
> would need to copy the data if sending to a different machine.
> 
> -- 
> Cameron Kerr <ckerr@REDACTED>
> Telecommunications Teaching Fellow
> University of Otago, New Zealand
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 

-- 
View this message in context: http://www.nabble.com/Microsoft-developer-vision---functional-programming-tf4646804.html#a13311055
Sent from the Erlang Questions mailing list archive at Nabble.com.



From exta7@REDACTED  Sat Oct 20 17:39:30 2007
From: exta7@REDACTED (Zvi)
Date: Sat, 20 Oct 2007 08:39:30 -0700 (PDT)
Subject: [erlang-questions] Microsoft developer vision - functional
 programming
In-Reply-To: <20071020040235.GA24943@bruce.local>
References: <4717562F.5050809@ericsson.com>
	<837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
	<20071020040235.GA24943@bruce.local>
Message-ID: <13311055.post@talk.nabble.com>


this would work if all the data stored in the global VM heap, but in Erlang
b/c of Soft Real-Time constraint, each process has it's own heap, so you
can't just pass pointer to object from another process' heap, which can be
garbage collected any time.
 Erlang have some optimizations, for large binaries, which are stored in the
global heap, and using reference counting for garbage collection.

Zvi


Cameron Kerr-2 wrote:
> 
> On Fri, Oct 19, 2007 at 10:42:42PM +0200, Hugh Perkins wrote:
> 
> Please correct me if I'm wrong, but isn't one of the benefits of
> Erlang's (in particular) single-assignment restriction that you only
> need to pass around a pointer, and never need to copy the data (because
> the data cannot be modified once it is first assigned). Obviously, you
> would need to copy the data if sending to a different machine.
> 
> -- 
> Cameron Kerr <ckerr@REDACTED>
> Telecommunications Teaching Fellow
> University of Otago, New Zealand
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 

-- 
View this message in context: http://www.nabble.com/Microsoft-developer-vision---functional-programming-tf4646804.html#a13311055
Sent from the Erlang Questions mailing list archive at Nabble.com.



From paul-trapexit@REDACTED  Sat Oct 20 19:20:51 2007
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Sat, 20 Oct 2007 10:20:51 -0700 (PDT)
Subject: [erlang-questions] Microsoft developer vision -
	functional	programming
In-Reply-To: <13311055.post@talk.nabble.com>
References: <4717562F.5050809@ericsson.com>
	<837db430710191342s294d3c52h3aa52fde78f9d806@mail.gmail.com>
	<20071020040235.GA24943@bruce.local> <13311055.post@talk.nabble.com>
Message-ID: <Pine.LNX.4.52L2.0710201019230.15501@herbie>

Perhaps this is a good time to mention the hybrid mode of the emulator?

http://user.it.uu.se/~kostis/Papers/toplas06.pdf

Depending upon your application, it might improve performance.

-- p

On Sat, 20 Oct 2007, Zvi wrote:

>
> this would work if all the data stored in the global VM heap, but in Erlang
> b/c of Soft Real-Time constraint, each process has it's own heap, so you
> can't just pass pointer to object from another process' heap, which can be
> garbage collected any time.
>  But Erlang have some optimizations, for large binaries, which are stored in
> the global heap, and using reference counting for garbage collection.
>
> Zvi
>
>
> Cameron Kerr-2 wrote:
> >
> > On Fri, Oct 19, 2007 at 10:42:42PM +0200, Hugh Perkins wrote:
> >
> > Please correct me if I'm wrong, but isn't one of the benefits of
> > Erlang's (in particular) single-assignment restriction that you only
> > need to pass around a pointer, and never need to copy the data (because
> > the data cannot be modified once it is first assigned). Obviously, you
> > would need to copy the data if sending to a different machine.
> >
> > --
> > Cameron Kerr <ckerr@REDACTED>
> > Telecommunications Teaching Fellow
> > University of Otago, New Zealand
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Microsoft-developer-vision---functional-programming-tf4646804.html#a13311055
> Sent from the Erlang Questions mailing list archive at Nabble.com.
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

Optimism is an essential ingredient of innovation. How else can the
individual favor change over security?

  -- Robert Noyce


From per@REDACTED  Sat Oct 20 21:02:38 2007
From: per@REDACTED (Per Hedeland)
Date: Sat, 20 Oct 2007 21:02:38 +0200 (CEST)
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <4719A77D.9090608@ghostgun.com>
Message-ID: <200710201902.l9KJ2cPw094706@pluto.hedeland.org>

jm <jeffm@REDACTED> wrote:
>
>it appears somthing is causing erlang to close stdin to the port,

Yes, that was pretty clear from your earlier results.

> but 
>I'm scratching my head as to why as no error is logged by the node.

Is the port owner process still running? The port gets automatically
closed when the owner exits.

--Per Hedeland


From eajam@REDACTED  Sun Oct 21 04:32:22 2007
From: eajam@REDACTED (Alex Alvarez)
Date: Sun, 21 Oct 2007 02:32:22 +0000
Subject: [erlang-questions] 64-bit erlang
In-Reply-To: <Pine.LNX.4.64.0710201032001.7841@speed.localdomain>
References: <Pine.LNX.4.64.0710201032001.7841@speed.localdomain>
Message-ID: <BLU111-W478BD5CA6D8361E919406AAD990@phx.gbl>

Good question...I have Ubuntu x64 on my other machine and it comes w/ Erlang, I believe, compiled to x64 (http://packages.ubuntu.com/gutsy/interpreters/erlang), so I have to conclude it's possible.
 
Cheers, Alex

> Date: Sat, 20 Oct 2007 10:38:25 +0530> From: saifi.khan@REDACTED> To: erlang-questions@REDACTED> Subject: [erlang-questions] 64-bit erlang> > Hi:> > Is it possible to compile a clean 64-bit version of erlang > from the source tar ball available on erlang.org ?> > There is a paper that does talk about porting Beam on AMD64.> > I am running a clean 64-bit AMD64 X2 box with a > LFS (Linux From Scratch) based Linux kernel 2.6.22 and gcc 4.1.x.> > Can some experienced erlangers share their insight on this ?> > Thanks in advance.> > thanks> Saifi.> > -------------------------------------------------------> TWINCLING Society http://www.twincling.org/> freedom of innovation Hyderabad AP, India.> -------------------------------------------------------> > > > _______________________________________________> erlang-questions mailing list> erlang-questions@REDACTED> http://www.erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/5edd713f/attachment.htm>

From eajam@REDACTED  Sun Oct 21 05:51:29 2007
From: eajam@REDACTED (Alex Alvarez)
Date: Sun, 21 Oct 2007 03:51:29 +0000
Subject: [erlang-questions] More nodes than machines
In-Reply-To: <D398AD75-56E6-4905-BD02-BC91DA1804C2@hypotheticalabs.com>
References: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>
	<D398AD75-56E6-4905-BD02-BC91DA1804C2@hypotheticalabs.com>
Message-ID: <BLU111-W20ED41DE8444903B6DFD15AD990@phx.gbl>

Interesting!  Thanks for the link!

> From: kevin@REDACTED> Date: Wed, 17 Oct 2007 10:37:30 -0400> To: jay@REDACTED> CC: erlang-questions@REDACTED> Subject: Re: [erlang-questions] More nodes than machines> > Bob was kind enough to answer many of these questions for me which > I've blogged about here:> > http://weblog.hypotheticalabs.com/?p=171> > --Kevin> On Oct 17, 2007, at 9:09 AM, Jay Nelson wrote:> > > This question is directed to Bob Ippolito because he said the> > following in the thread on "OCaml vs Erlang":> >> >> A year later we have about 16 machines running 80 Erlang nodes> > powering about 16> >> different "components" of our infrastructure> >> > 1) Why did you choose to use multiple nodes on each physical machine?> >> > - Are your machines multicore?> > - Are you using SMP for your erlang nodes?> > - Are you using virtualized machines (via Xen, VMWare or other> > means)?> > - What kind of machines are you running (CPU and memory)?> >> > 2) Any performance numbers on simultaneous connections or connection> > rate per minute sustainable by your mochiweb http server?> >> >> > jay> >> > _______________________________________________> > erlang-questions mailing list> > erlang-questions@REDACTED> > http://www.erlang.org/mailman/listinfo/erlang-questions> > _______________________________________________> erlang-questions mailing list> erlang-questions@REDACTED> http://www.erlang.org/mailman/listinfo/erlang-questions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/ac7d42c1/attachment.htm>

From vinoski@REDACTED  Sun Oct 21 07:40:38 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Sun, 21 Oct 2007 01:40:38 -0400
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
Message-ID: <65b2728e0710202240k65502ea1v881a045666ee9668@mail.gmail.com>

On 10/20/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> Hi
> I am playing around with Tim Bray's widefinder problem.
> Actually I am playing with Steve V's latest version
> see http://steve.vinoski.net/blog/2007/10/18/ok-just-one-more-wf/
>
> In it he is using a dict for Boyer-Moore searching, the dict is just
> a map from char code to shift amount i.e. the keys are 1 - 255
> and the values are ints.
> I figured that it would be faster to just have a tuple with the shift
> values.
> So I changed the code to generate a tuple instead of a dict. And the
> lookups changed from
> dict:fetch(C1, Tbl)
> to
> element(C1,Tbl)
>
> To my big surprise the program now runs slower.
> On a file with 1M rows approx. 192 M.
>
> Steve's original code, using dics, as reported by time
> real    0m20.444s
> user    0m37.718s
> sys     0m0.612s
>
> My modified, using a tuple
> real    0m25.849s
> user    0m44.203s
> sys     0m3.332s
>
> Interesting to note is the high value for sys in my version.
> Also the tuple version does not manage to max out the CPU use
> it peaks at ~85%, while the dict version gets 100% CPU load.
> The program starts 64 erlang processes on my dual core laptop.
>
> My only idea is that this is caused by some pathological CPU cache
> behaviour. Does anyone have a better explanation?
>

Hi Anders, I don't know the answer, but I agree that it seems strange. I
wrote some tests that did simple lookups against dicts and tuples in a loop
and timed them, and tuples were faster, but when I used them in my Wide
Finder code, they were slower, just as you saw.

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/9f6a1791/attachment.htm>

From bob@REDACTED  Sun Oct 21 08:09:48 2007
From: bob@REDACTED (Bob Ippolito)
Date: Sat, 20 Oct 2007 23:09:48 -0700
Subject: [erlang-questions] More nodes than machines
In-Reply-To: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>
References: <7EE402AD-2103-4E0E-AB4F-0D3DF38BB447@duomark.com>
Message-ID: <6a36e7290710202309i7a6ae5e6n9eb234f5c51d638b@mail.gmail.com>

For the most part we use a lot of nodes because we start one node per
component of the system. It has more to do with ease of deployment and
management than anything else.

As far as mochiweb goes we definitely aren't bound by it's
limitations. It's as least as good as any other pure erlang http
server that takes advantage of the VM's header parsing. Ours all sit
behind nginx so there are not massive simultaneous connections for our
typical uses, though it should be good at that too.

-bob

On 10/17/07, Jay Nelson <jay@REDACTED> wrote:
> This question is directed to Bob Ippolito because he said the
> following in the thread on "OCaml vs Erlang":
>
>  > A year later we have about 16 machines running 80 Erlang nodes
> powering about 16
>  > different "components" of our infrastructure
>
> 1) Why did you choose to use multiple nodes on each physical machine?
>
>      - Are your machines multicore?
>      - Are you using SMP for your erlang nodes?
>      - Are you using virtualized machines (via Xen, VMWare or other
> means)?
>      - What kind of machines are you running (CPU and memory)?
>
> 2) Any performance numbers on simultaneous connections or connection
> rate per minute sustainable by your mochiweb http server?
>
>
> jay
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bob@REDACTED  Sun Oct 21 11:17:39 2007
From: bob@REDACTED (Bob Ippolito)
Date: Sun, 21 Oct 2007 02:17:39 -0700
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <65b2728e0710202240k65502ea1v881a045666ee9668@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
	<65b2728e0710202240k65502ea1v881a045666ee9668@mail.gmail.com>
Message-ID: <6a36e7290710210217i2e116ab9ka442f5f4a1483553@mail.gmail.com>

In my experience using tuples for this purpose, it seems like in many
or most cases tuples are instantiated from scratch every function call
even though they're constant in the bytecode. You might just be
assuming some optimization that Erlang doesn't actually do. This is
the sort of thing that the hybrid model might fix, but as far as I
know the only data structures that are on a global heap is large
binaries, not tuples of any kind or size. I'm mostly just guessing
though, because I have not read any of the Erlang VM's source (and
it's great that I haven't had to!).

You may have better luck restructuring your code in such a way that
this can't happen, either by generating a function that is the
equivalent of element(N, Tuple), e.g. a function with 256 clauses, or
by keeping the tuple in the state of a process (maybe several
processes) so that it doesn't get continuously allocated and
deallocated.

-bob

On 10/20/07, Steve Vinoski <vinoski@REDACTED> wrote:
>
> On 10/20/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > Hi
> > I am playing around with Tim Bray's widefinder problem.
> > Actually I am playing with Steve V's latest version
> > see http://steve.vinoski.net/blog/2007/10/18/ok-just-one-more -wf/
> >
> > In it he is using a dict for Boyer-Moore searching, the dict is just
> > a map from char code to shift amount i.e. the keys are 1 - 255
> > and the values are ints.
> > I figured that it would be faster to just have a tuple with the shift
> values.
> > So I changed the code to generate a tuple instead of a dict. And the
> > lookups changed from
> > dict:fetch(C1, Tbl)
> > to
> > element(C1,Tbl)
> >
> > To my big surprise the program now runs slower.
> > On a file with 1M rows approx. 192 M.
> >
> > Steve's original code, using dics, as reported by time
> > real    0m20.444s
> > user    0m37.718s
> > sys     0m0.612s
> >
> > My modified, using a tuple
> > real    0m25.849s
> > user    0m44.203s
> > sys     0m3.332s
> >
> > Interesting to note is the high value for sys in my version.
> > Also the tuple version does not manage to max out the CPU use
> > it peaks at ~85%, while the dict version gets 100% CPU load.
> > The program starts 64 erlang processes on my dual core laptop.
> >
> > My only idea is that this is caused by some pathological CPU cache
> > behaviour. Does anyone have a better explanation?
> >
>
> Hi Anders, I don't know the answer, but I agree that it seems strange. I
> wrote some tests that did simple lookups against dicts and tuples in a loop
> and timed them, and tuples were faster, but when I used them in my Wide
> Finder code, they were slower, just as you saw.
>
> --steve
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bjorn@REDACTED  Sun Oct 21 11:41:12 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 21 Oct 2007 11:41:12 +0200
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
Message-ID: <o1y7dwvm87.fsf@erix.ericsson.se>

"Anders Nygren" <anders.nygren@REDACTED> writes:
> 
> My only idea is that this is caused by some pathological CPU cache
> behaviour. Does anyone have a better explanation?

Could be. I have another GUESS: using a tuple instead of dict will
result in a smaller heap size for the spawned process, which will lead
to more frequent garbage collections. You could use the spawn_opt()
BIF with the min_heap_size option.

To get a similar heap size as when using a dict, use

        erts_debug:size(wfbm3:init()).

which is 1120.

/Bjorn

-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From bjorn@REDACTED  Sun Oct 21 11:48:14 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 21 Oct 2007 11:48:14 +0200
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <6a36e7290710210217i2e116ab9ka442f5f4a1483553@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
	<65b2728e0710202240k65502ea1v881a045666ee9668@mail.gmail.com>
	<6a36e7290710210217i2e116ab9ka442f5f4a1483553@mail.gmail.com>
Message-ID: <o1tzokvlwh.fsf@erix.ericsson.se>

"Bob Ippolito" <bob@REDACTED> writes:

> In my experience using tuples for this purpose, it seems like in many
> or most cases tuples are instantiated from scratch every function call
> even though they're constant in the bytecode. You might just be
> assuming some optimization that Erlang doesn't actually do. This is
> the sort of thing that the hybrid model might fix, but as far as I
> know the only data structures that are on a global heap is large
> binaries, not tuples of any kind or size. I'm mostly just guessing
> though, because I have not read any of the Erlang VM's source (and
> it's great that I haven't had to!).

Correct for all releases up to and including R11B-5.

> You may have better luck restructuring your code in such a way that
> this can't happen, either by generating a function that is the
> equivalent of element(N, Tuple), e.g. a function with 256 clauses, or
> by keeping the tuple in the state of a process (maybe several
> processes) so that it doesn't get continuously allocated and
> deallocated.

That is good advice in general (up to and including all R11B releases),
but in this case I don't think that a constant tuple was used (i.e. the
tuple is already part of the state of a process).

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From kosik@REDACTED  Sun Oct 21 11:47:18 2007
From: kosik@REDACTED (Matej Kosik)
Date: Sun, 21 Oct 2007 11:47:18 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <8209f740710200652l5c4ba58n396f51329d868e76@mail.gmail.com>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
	<8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
	<4719FA75.3040707@fiit.stuba.sk>
	<8209f740710200652l5c4ba58n396f51329d868e76@mail.gmail.com>
Message-ID: <471B2026.80405@fiit.stuba.sk>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ulf Wiger wrote:
> 2007/10/20, Matej Kosik <kosik@REDACTED>:
>> Doesn't shadowing of meanings of variables in Erlang similarly
>> complicate reasoning? (in one position some variable has some
>> meaning and elsewhere in the term suddenly has different meaning).
> 
> Well, it could, but in most cases where it would be confusing, the
> compiler will give you a warning.
> 
> Personally, I try to use some naming convention for variables to
> minimize the risk of confusion.
> 
> 
>> Non-functional features of Erlang are present in its fundamental constructs:
>> - - send
>> - - receive
>> - - spawn
>> These cannot be modelled in (purely) functional languages, can they? How?
> 
> Almost by definition, the answer is "no", I think?  (:

Thank you for the answers,

Hm. Today I have look at the Wikipedia article and it (at the begining) says:

"The sequential subset of Erlang is a functional language".

This is correct.

However, Joe's new book says:

"Erlang is a functional programming language and has nonmutable state."
(page 32, at the top)

Isn't this false? With `spawn', `send' and `receive' you can model mutable state---the state of
processes changes over time and this is very useful. The change of the state can be observed from
the outside. Isn't this contradictory to the reality?

I agree that fragments of code written in Erlang's functional subset can be subject to
proof-techniques (about program correctness) developed for functional programs but what about the
non-functional constructs? Have you developed some proof techniques to cover also these constructs?
Which? I am interested. (I do not mean crashes).

> 
> (I guess I should let the real language experts answer this, but it's Saturday,
> and I don't have anything better to do right now. I trust I'll get my fingers
> slapped if I write something stupid).
> 
> At least it's difficult. Haskell uses 'monads', to clearly mark where the code
> has side-effects. This allows the compiler to trust that all code outside of
> the monad is pure. Other languages can use the fact that these operations
> are contained within the runtime, and can optimize them. Felix* does this,
> for example; I don't know about Concurrent ML. I believe that when Felix
> participated in the Language Shootout, its compiler reduced the concurrency
> benchmark down to a function that did nothing**. This doesn't mean that
> Felix models "fibres" as being pure, but as long as the effect of the
> impurity is understood, a smart compiler can sometimes eliminate it,
> or at least simplify it significantly. I guess this is sort of a grey area. (:
> 
> It is partly connected to the fact that most of the surroundings are pure,
> and very well understood. Erlang can, for example, decide to pass a
> pointer rather than copy a message, since there are no user-level
> operations that can mess with the low-level representation of the
> message. There was once a paper suggesting that a gen_server:call()
> could be reduced to a function call, under certain circumstances
> (http://www.erlang.se/workshop/2002/Stenman.pdf), but it
> suggested dynamic checking to ensure that the conditions were met,
> since it would be difficult to do statically in Erlang.
> 
> (OHaskell not only performs this check statically, it enforces it:
> http://www.cs.chalmers.se/~nordland/ohaskell/)
> 
> Erlang's approach is fairly pragmatic. It has basically tried to make sure
> that all non-pure operations can be modeled using spawn, send and receive,
> but the only parts of an erlang program that the compiler can trust to be
> free from side-effects are clause heads, the left hand side of an expression
> (pattern matching), and guard expressions. Every (non-guard) function call
> can potentially result in a side-effect. Some BIFs are also known to be pure
> and can be optimized by the compiler.
> 
> * http://www.felix-lang.org - a OCaml/C++ hybrid of sorts
> ** This Felix entry was disqualified, which caused some debate. Google
> will tell you more.
> 
> BR,
> Ulf W
> 

Best regards
- --
Matej Ko??k
ICQ: 300133844
skype: matej_kosik
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHGyAmL+CaXfJI/hgRAsxxAJ4hFB3JPFbfTu0NSRhyp/9kIY8OBgCfSxUd
UjmucR1QFVgotr5GLqtOhn8=
=GhZY
-----END PGP SIGNATURE-----


From ulf@REDACTED  Sun Oct 21 12:30:40 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Sun, 21 Oct 2007 12:30:40 +0200
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <471B2026.80405@fiit.stuba.sk>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
	<8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
	<4719FA75.3040707@fiit.stuba.sk>
	<8209f740710200652l5c4ba58n396f51329d868e76@mail.gmail.com>
	<471B2026.80405@fiit.stuba.sk>
Message-ID: <8209f740710210330i6216d62dy6cbeb68c50ebf1d3@mail.gmail.com>

2007/10/21, Matej Kosik <kosik@REDACTED>:
>
> Hm. Today I have look at the Wikipedia article and it (at the begining) says:
>
> "The sequential subset of Erlang is a functional language".
>
> This is correct.
>
> However, Joe's new book says:
>
> "Erlang is a functional programming language and has nonmutable state."
> (page 32, at the top)
>
> Isn't this false? With `spawn', `send' and `receive' you can model mutable state---the state of
> processes changes over time and this is very useful. The change of the state can be observed from
> the outside. Isn't this contradictory to the reality?

Within each process, state is nonmutable. Observing from the outside,
it appears as if the state of a process is changing, and processes can
communicate with each other through messages; they cannot alter the
state of another process (short of killing it).

If you take a broader view of 'state', to include also the contents of
the mailbox, then it is possible to alter the state of another process. (:


> I agree that fragments of code written in Erlang's functional subset can be subject to
> proof-techniques (about program correctness) developed for functional programs but what about the
> non-functional constructs? Have you developed some proof techniques to cover also these constructs?
> Which? I am interested. (I do not mean crashes).

Within limits, erlang code can be converted to uCrl, which can then be used
for model checking. There is another approach that post-processes trace
output from a running system, abstracting away detail from the trace, and
then e.g. model checking the abstract trace. There are some other approaches
as well.

BR,
Ulf W


From tobias.gerdin@REDACTED  Sun Oct 21 15:04:34 2007
From: tobias.gerdin@REDACTED (Tobias Gerdin)
Date: Sun, 21 Oct 2007 15:04:34 +0200
Subject: [erlang-questions]  Why single-assignment with non-shared state?
References: <11430FFE-78DE-4BCF-A2C0-F264547BDFB3@gmail.com>
Message-ID: <233F8849-5AF9-44A9-BBA4-6EB557B49DEB@gmail.com>

But in Erlang since state is not shared between processes you would  
never get race conditions anyway. Single-assignment makes a lot of  
sense when engaging in shared-memory thread-based programming, but  
since that is not the case in Erlang I was curious of the philosophy  
behind single-assignment in Erlang's case.

On 20 oktober 2007, at 13.37, Cameron Kerr wrote:

> If you can only assign to a variable once, you cannot have race  
> conditions.
>
> On 20/10/2007, at 10:50 PM, Tobias Gerdin wrote:
>
>> Hi there,
>>
>> I do not understand why the single-assignment restriction exists in
>> Erlang (even though I very much enjoy functional programming) given
>>> that processes do not share state. What is the problem with mutable
>>> state in a concurrent system when other processes cannot access  
>>> other
>>> processes' state?
>>>
>>> I was hoping someone could enlighten me on this matter.
>>>
>>>
>>> Regards,
>>>
>>> Tobias



From tobias.gerdin@REDACTED  Sun Oct 21 15:05:13 2007
From: tobias.gerdin@REDACTED (Tobias Gerdin)
Date: Sun, 21 Oct 2007 15:05:13 +0200
Subject: [erlang-questions] Fwd: Why single-assignment with non-shared state?
References: <54588F09-37BD-4136-BA7C-CD2433EBAC7B@gmail.com>
Message-ID: <6A31E2B2-FD58-43B9-A5C8-D0A90C57094D@gmail.com>

Well as long as the expressions you match on doesn't change during  
matching I do not see the problem. There are several prominent  
languages such as the ML-family, Scala, or even Lisps which feature  
both pattern matching and assignment.

On 20 oktober 2007, at 13.33, Ulf Wiger wrote:

> While single assignment and pattern matching go very well
> together, it's a little bit less obvious how mutable state and
> pattern matching would combine as nicely. Erlang's current
> semantics for binding and matching make it quite straight-
> forward to reason about even very complex programming
> patterns.
>
> BR,
> Ulf W
>
> 2007/10/20, Tobias Gerdin <tobias.gerdin@REDACTED>:
>> Hi there,
>>
>> I do not understand why the single-assignment restriction exists in
>> Erlang (even though I very much enjoy functional programming) given
>> that processes do not share state. What is the problem with mutable
>> state in a concurrent system when other processes cannot access other
>> processes' state?
>>
>> I was hoping someone could enlighten me on this matter.
>>
>>
>> Regards,
>>
>> Tobias
>>
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>




From S.J.Thompson@REDACTED  Sun Oct 21 16:43:25 2007
From: S.J.Thompson@REDACTED (S.J.Thompson)
Date: Sun, 21 Oct 2007 15:43:25 +0100 (BST)
Subject: [erlang-questions] Interested in writing a review of "Programming
	Erlang"?
Message-ID: <Pine.GSO.4.58.0710211540170.11887@myrtle.ukc.ac.uk>


If you would be interested in writing a review of Joe Armstrong's
"Programming Erlang" for the Journal of Functional Programming, then
perhaps you could contact me?

Guidelines about writing book reviews for JFP can be found at

  http://www.cambridge.org/journals/JFP/reviews.asp

Thanks very much,

Simon Thompson




From vinoski@REDACTED  Sun Oct 21 17:12:19 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Sun, 21 Oct 2007 11:12:19 -0400
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <o1y7dwvm87.fsf@erix.ericsson.se>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
	<o1y7dwvm87.fsf@erix.ericsson.se>
Message-ID: <65b2728e0710210812v3e5a6cd1w3dc7906bcd67fd92@mail.gmail.com>

On 21 Oct 2007 11:41:12 +0200, Bjorn Gustavsson <bjorn@REDACTED>
wrote:
>
> "Anders Nygren" <anders.nygren@REDACTED> writes:
> >
> > My only idea is that this is caused by some pathological CPU cache
> > behaviour. Does anyone have a better explanation?
>
> Could be. I have another GUESS: using a tuple instead of dict will
> result in a smaller heap size for the spawned process, which will lead
> to more frequent garbage collections. You could use the spawn_opt()
> BIF with the min_heap_size option.
>
> To get a similar heap size as when using a dict, use
>
>         erts_debug:size(wfbm3:init()).
>
> which is 1120.
>

I didn't try this or Bob's suggestion of simulating a tuple, but I did
eliminate a dict lookup in the code that finds shift values, and also
changed the macros to use hard-coded string lengths (since the strings are
fixed) rather than recalculating them with length/1, and got another 25%
speedup:

<http://steve.vinoski.net/blog/2007/10/21/faster-wf-still/>


--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/0f44aac1/attachment.htm>

From thomasl_erlang@REDACTED  Sun Oct 21 17:21:23 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Sun, 21 Oct 2007 08:21:23 -0700 (PDT)
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <233F8849-5AF9-44A9-BBA4-6EB557B49DEB@gmail.com>
Message-ID: <82847.49893.qm@web38807.mail.mud.yahoo.com>


--- Tobias Gerdin <tobias.gerdin@REDACTED> wrote:

> But in Erlang since state is not shared between
> processes you would  
> never get race conditions anyway. Single-assignment
> makes a lot of  
> sense when engaging in shared-memory thread-based
> programming, but  
> since that is not the case in Erlang I was curious
> of the philosophy  
> behind single-assignment in Erlang's case.

If you have pattern matching, you can write something
like this:

member(X, [X|Xs]) -> true;
member(X, [_|Ys]) -> member(X, Ys);
member(X, _) -> false.

Consider the first clause. What should be done when
you encounter X twice?(*)

- disallow it by refusing to compile (do not permit
"non-linear patterns")

- interpret it as comparing the two occurrences for
equality

- first assign X the value of argument 1, then the
value of the head of argument 2 (probably not what you
want)

Some languages (ML, Haskell?) chose option 1. Some
languages (Prolog, Miranda, Erlang) chose option 2. 

As an aside, note that many functional languages
permit "reassigning" a variable by introducing a new
scope. You can "reassign" x by writing something like
this:

 let x = 3 in
  let x = 42+x in
   2*x

which would return 90. (However, this is not real
assignment, it just uses the property that variable
names can be changed.) Anyway, doing this in Erlang
requires somewhat contorted code, but it's possible
and, indeed, sometimes indispensable.

Best,
Thomas

(*) As an extra complication, there normally is no
restriction on the order of pattern matches. Thus, the
compiler could choose to begin compiling member/2 by
matching argument 2 or argument 1 and get somewhat
different code in the end. The compiler can and will
(and should!) exploit this to optimize pattern
matching. 

In Erlang, binary patterns break this "order
independent" property and _do_ have some restrictions.
But apart from that, you can think of pattern matches
as done in any arbitrary order.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From anders.nygren@REDACTED  Sun Oct 21 18:46:47 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Sun, 21 Oct 2007 11:46:47 -0500
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <65b2728e0710210812v3e5a6cd1w3dc7906bcd67fd92@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
	<o1y7dwvm87.fsf@erix.ericsson.se>
	<65b2728e0710210812v3e5a6cd1w3dc7906bcd67fd92@mail.gmail.com>
Message-ID: <d3520a2f0710210946l3d358b61l45af3d66eb94e8a4@mail.gmail.com>

On 10/21/07, Steve Vinoski <vinoski@REDACTED> wrote:
> On 21 Oct 2007 11:41:12 +0200, Bjorn Gustavsson <bjorn@REDACTED>
> wrote:
> > "Anders Nygren" <anders.nygren@REDACTED> writes:
> > >
> > > My only idea is that this is caused by some pathological CPU cache
> > > behaviour. Does anyone have a better explanation?
> >
> > Could be. I have another GUESS: using a tuple instead of dict will
> > result in a smaller heap size for the spawned process, which will lead
> > to more frequent garbage collections. You could use the spawn_opt()
> > BIF with the min_heap_size option.
> >
> > To get a similar heap size as when using a dict, use
> >
> >         erts_debug:size(wfbm3:init()).
> >
> > which is 1120.
> >
>
> I didn't try this or Bob's suggestion of simulating a tuple, but I did
> eliminate a dict lookup in the code that finds shift values, and also
> changed the macros to use hard-coded string lengths (since the strings are
> fixed) rather than recalculating them with length/1, and got another 25%
> speedup:
>
> <http://steve.vinoski.net/blog/2007/10/21/faster-wf-still/>
>
>

So I then took Steve's latest and
- changed to using a tuple instead of dict and VICTORY :)


Steve's tbray16 on my laptop
real    0m10.929s
user    0m18.477s
sys     0m0.560s

With tuple instead of dict
real    0m14.126s
user    0m22.373s
sys     0m1.924s


- added [{min_heap_size, 3000}] to the spawn, as suggested by Bjorn

With tuple and min_heap_size, 1000
real    0m7.358s
user    0m11.725s
sys     0m0.480s

With tuple and min_heap_size, 2000
real    0m6.760s
user    0m10.957s
sys     0m0.388s

With tuple and min_heap_size, 3000
real    0m6.305s
user    0m10.149s
sys     0m0.380s

After that increasing the heap size does not make any difference.

So my next step will be to play with garbage collection and see what I can get.

/Anders


From vinoski@REDACTED  Sun Oct 21 22:12:20 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Sun, 21 Oct 2007 16:12:20 -0400
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <d3520a2f0710210946l3d358b61l45af3d66eb94e8a4@mail.gmail.com>
References: <d3520a2f0710200708r1a1ae503i65ddfd3509153873@mail.gmail.com>
	<o1y7dwvm87.fsf@erix.ericsson.se>
	<65b2728e0710210812v3e5a6cd1w3dc7906bcd67fd92@mail.gmail.com>
	<d3520a2f0710210946l3d358b61l45af3d66eb94e8a4@mail.gmail.com>
Message-ID: <65b2728e0710211312g786909f6l8c2e6fd7760015d@mail.gmail.com>

On 10/21/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> On 10/21/07, Steve Vinoski <vinoski@REDACTED> wrote:
> > On 21 Oct 2007 11:41:12 +0200, Bjorn Gustavsson <bjorn@REDACTED>
> > wrote:
> > > "Anders Nygren" <anders.nygren@REDACTED> writes:
> > > >
> > > > My only idea is that this is caused by some pathological CPU cache
> > > > behaviour. Does anyone have a better explanation?
> > >
> > > Could be. I have another GUESS: using a tuple instead of dict will
> > > result in a smaller heap size for the spawned process, which will lead
> > > to more frequent garbage collections. You could use the spawn_opt()
> > > BIF with the min_heap_size option.
> > >
> > > To get a similar heap size as when using a dict, use
> > >
> > >         erts_debug:size(wfbm3:init()).
> > >
> > > which is 1120.
> > >
> >
> > I didn't try this or Bob's suggestion of simulating a tuple, but I did
> > eliminate a dict lookup in the code that finds shift values, and also
> > changed the macros to use hard-coded string lengths (since the strings
> are
> > fixed) rather than recalculating them with length/1, and got another 25%
> > speedup:
> >
> > <http://steve.vinoski.net/blog/2007/10/21/faster-wf-still/>
> >
> >
>
> So I then took Steve's latest and
> - changed to using a tuple instead of dict and VICTORY :)


<snip/>

Anders sent me his code and I ran it on my 8-core Linux box, with the
following performance results. VICTORY is right! :-)

real    0m1.904s
user    0m7.917s
sys     0m1.185s

Like I mentioned to Anders in private email, it's nice to have someone more
experienced with Erlang finally taking a look at this; I'm still a relative
newbie.
One thing I've liked about this entire exercise is that the early attempts
at solving Tim Bray's Wide Finder in Erlang were taking minutes to execute
and were providing only partial answers. Several of us then started
whittling away at it, and because of the richness of the language, we had a
variety of different avenues to explore. Over time, we've vastly increased
the performance of our solutions. Anders's solution now beats Ruby on the
same machine by about 0.3s, and because of the way it uses multiple cores,
it will likely execute extremely quickly and efficiently when Tim gets a
chance to try it on his T5120.

Yes, fast solutions in other languages were quickly found, but those had
almost nowhere to go beyond their initial forms in terms of improvement, not
because they were already so fast, but because the languages ran out of
alternatives. This is especially true when it comes to taking advantage of
the T5120's many cores. I'm a fan of many languages, including Ruby, Python,
Perl, and C++, all of which have figured prominently in the collection of
various Wide Finder solutions. But for my money, Erlang has fulfilled Tim's
original wishes the best, which is to take the best possible advantage of
all those cores.

Anders, I look forward to even more speed when you finish those GC tweaks!
:-)

thanks,
--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/efb13931/attachment.htm>

From aaron.feng@REDACTED  Mon Oct 22 00:34:45 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Sun, 21 Oct 2007 18:34:45 -0400
Subject: [erlang-questions] Programming Erlang book code question
Message-ID: <4e3934860710211534p5e64f03cmdc269e331c973fcb@mail.gmail.com>

The following code snippet appeared on page 352:

-module(sellaprime_supervisor).
-behaviour(supervisor). % see erl -man supervisor
-export([start/0, start_in_shell_for_testing/0, start_link/1, init/1]).

start() -> spawn(fun() ->
                        supervisor:start_link({local,?MODULE}, ?MODULE, _Arg
= [])
                        end).

start_in_shell_for_testing() ->
  {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = []),
  unlink(Pid).

%% rest of the code intentionally left out

Question 1:

What's the benefit to call spawn for supervisor:start_link in the start()
function?  Why not just call supervisor:start_link without the spawn?
Doesn't supervisor:start_link already spawn and register the process
already?

Question 2:

What's the benefit for start_in_shell_for_testing() function?  Why not just
call start() function when testing in the shell?  I know what the
unlink(Pid) does, but why unlinking it?  What if unlink it left out?

Question 3:

I don't understand _Arg = [] ?  why not just pass [] as the last argument of
start_link?

Thanks in advance,

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/31de8572/attachment.htm>

From ckerr@REDACTED  Mon Oct 22 00:52:55 2007
From: ckerr@REDACTED (Cameron Kerr)
Date: Mon, 22 Oct 2007 11:52:55 +1300
Subject: [erlang-questions] Why single-assignment with non-shared state?
In-Reply-To: <471B2026.80405@fiit.stuba.sk>
References: <CD41EA92-999F-45ED-B885-27DC349ABB7D@gmail.com>
	<8209f740710200433n4c54d6b9h8f05fe9fb13af51e@mail.gmail.com>
	<4719FA75.3040707@fiit.stuba.sk>
	<8209f740710200652l5c4ba58n396f51329d868e76@mail.gmail.com>
	<471B2026.80405@fiit.stuba.sk>
Message-ID: <B1A3ED12-50CD-419B-B911-05C626CA10AC@cs.otago.ac.nz>


On 21/10/2007, at 10:47 PM, Matej Kosik wrote:

> I agree that fragments of code written in Erlang's functional  
> subset can be subject to
> proof-techniques (about program correctness) developed for  
> functional programs but what about the
> non-functional constructs? Have you developed some proof techniques  
> to cover also these constructs?
> Which? I am interested. (I do not mean crashes).

Concurrent models such as CSP could probably be applied to the  
concurrent parts of an Erlang program, though I have yet to read the  
CSP book.

-- 
? Cameron Kerr  ? ? ckerr@REDACTED ?
? ?
? Telecommunications Teaching Fellow & SysAdmin ?
? ? http://humbledown.org/blog/ ? ? New: 027 7175 244 ?

"Technological advances are not made by sadomasochistic, cultic, tool- 
worshipping pain freaks." ? http://freeshells.ch/~revence/myths.txt




From aaron.feng@REDACTED  Mon Oct 22 04:22:26 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Sun, 21 Oct 2007 22:22:26 -0400
Subject: [erlang-questions] Programming Erlang book code question
In-Reply-To: <aec8d6fc0710211558u42620dc9q379396a7918fd2c6@mail.gmail.com>
References: <4e3934860710211534p5e64f03cmdc269e331c973fcb@mail.gmail.com>
	<aec8d6fc0710211558u42620dc9q379396a7918fd2c6@mail.gmail.com>
Message-ID: <4e3934860710211922wa062ebeh9033331afb877c2d@mail.gmail.com>

Hi Mateusz,

For question 2, appearently if any worker process dies, it will also cause
the supervisor to be killed.  That's why the unlink is there.  For example,
if the supervisor process is started, and you call a worker process with
incorrect number of arguments it will cause the supervisor to be killed.

You are right about question 3, it makes a lot of sense now.

For question 1, I tried out the code from the book, and I still don't
understand why the spawn is there fully.  You could be right about the
quicker initialization, but I think there's another reason for it.  If I
call the start() from the shell, I can't access the worker process that is
managed by the supervisor.  In fact, I have no idea how I can access the
worker process at all.  However if I call start_in_shell_for_testing() in
the shell, I can access the worker process like usual.

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071021/7e53deb4/attachment.htm>

From jeffm@REDACTED  Mon Oct 22 07:13:57 2007
From: jeffm@REDACTED (jm)
Date: Mon, 22 Oct 2007 15:13:57 +1000
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710201902.l9KJ2cPw094706@pluto.hedeland.org>
References: <200710201902.l9KJ2cPw094706@pluto.hedeland.org>
Message-ID: <471C3195.6090602@ghostgun.com>

Per Hedeland wrote:
> jm <jeffm@REDACTED> wrote:
>> it appears somthing is causing erlang to close stdin to the port,
> 
> Yes, that was pretty clear from your earlier results.

Just had to prove it.

>> but 
>> I'm scratching my head as to why as no error is logged by the node.
> 
> Is the port owner process still running? The port gets automatically
> closed when the owner exits.

In cleaning up the code I'd introduced another bug which is what the 
output in the email message was generated by. It was caused be me 
leaving out a return(0) after the part which generated the response for 
the list command. This caused it to fall though after sending the 
correct response and execute


int process_command(const char *command, char *buf, int *index, note_t 
*nlist) {
   ei_x_buff  result;
.....

/* Prepare the output buffer that will hold {ok, Result} or {error, 
Reason} */
   if (ei_x_new_with_version(&result) ||
       ei_x_encode_tuple_header(&result, 2) ||
       ei_x_encode_atom(&result, "error") ||
       ei_x_encode_atom(&result, "unsupported_command"))
     return(-1);

   write_cmd(&result);
   ei_x_free(&result);
   return(0);
} /* end of function */


It does however make me further suspect that it's the event messages, 
those not in response to commands, which the port is sending which is 
causing the problem.


Rather than post snipets of code I've made it available at

  http://ghostgun.com/software/erlang/inotify/notify_play_buggy.tar.bz2

I've been resisting this as I'd like to get it to work myself, worse I 
think the bug is either starring me in the face or is a gap in my 
knowledge of how to write an erlang port. Most of the code is straight 
forward there's nothing really tricky anywhere. Second, this port/module 
is Linux specific as it uses the inotify kernel interface. To run it 
you'll need a Linux box with a recent kernel (2.6.13 or later, I'm using 
2.6.21 on a debian box).

The way I've been testing it is to run the follow in the erlang shell 
from within the base directory

 > inotify:start().
 > inotify:open().
 > inotify:list().
 > inotify:add(3, "file", all)

Then in another window cd to the base directory and

$ cat file

file is actually an empty file in the base directory. This triggers two 
events open and close_nowrite. This is were it crashes. If you get 
further than this your doing better than me.

As far as licensing is concerned, in case it matters to anyone reading 
this, it's planned to release this under the Erlang Public License, 
LGPL, or something similar once this is working.


Jeff.



From thomasl_erlang@REDACTED  Mon Oct 22 09:48:32 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 22 Oct 2007 00:48:32 -0700 (PDT)
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <65b2728e0710211312g786909f6l8c2e6fd7760015d@mail.gmail.com>
Message-ID: <643517.43588.qm@web38804.mail.mud.yahoo.com>


--- Steve Vinoski <vinoski@REDACTED> wrote:

> Anders sent me his code and I ran it on my 8-core
> Linux box, with the
> following performance results. VICTORY is right! :-)
> 
> real    0m1.904s
> user    0m7.917s
> sys     0m1.185s
> 
> Like I mentioned to Anders in private email, it's
> nice to have someone more
> experienced with Erlang finally taking a look at
> this; I'm still a relative
> newbie.
> One thing I've liked about this entire exercise is
> that the early attempts
> at solving Tim Bray's Wide Finder in Erlang were
> taking minutes to execute
> and were providing only partial answers. Several of
> us then started
> whittling away at it, and because of the richness of
> the language, we had a
> variety of different avenues to explore. Over time,
> we've vastly increased
> the performance of our solutions. Anders's solution
> now beats Ruby on the
> same machine by about 0.3s, and because of the way
> it uses multiple cores,
> it will likely execute extremely quickly and
> efficiently when Tim gets a
> chance to try it on his T5120.
> 
> Yes, fast solutions in other languages were quickly
> found, but those had
> almost nowhere to go beyond their initial forms in
> terms of improvement, not
> because they were already so fast, but because the
> languages ran out of
> alternatives. This is especially true when it comes
> to taking advantage of
> the T5120's many cores. I'm a fan of many languages,
> including Ruby, Python,
> Perl, and C++, all of which have figured prominently
> in the collection of
> various Wide Finder solutions. But for my money,
> Erlang has fulfilled Tim's
> original wishes the best, which is to take the best
> possible advantage of
> all those cores.

Well done, everyone. You've chewed this over pretty
well, I'd say, and it has been interesting to see how
things have improved over time. Here are a couple of
thoughts on further improvements:

1. Native code compilation? It's a bit hit-and-miss,
but this could be the sort of problem that gains from
it.

2. The speedup is 4.15 on 8 cores (if I'm reading
things right: user/real). What is the bottleneck? Too
small input, too much I/O, or is there something that
could be improved or tuned further?

And for language fans, the language itself could be a
bit more helpful. While we haven't emphasized regexp
crunching, it still seems like things could be easier.
Here are some quick thoughts.

0. Not having to write the Boyer-Moore stuff by hand

1. Working with binaries like strings should be easier

2. Reading lines from files

3. Appropriate data chunking for file processing
(people tried all from a few KB to several MB per
chunk -- could the system figure out an appropriate
size on its own?)

4. Perhaps a streaming interface would be even better?
Jay Nelson suggested one a few years ago.

5. When looking at the use of dictionaries, it struck
me that ets:update_counter/3 could have avoided the
use of dictionaries and merging altogether. But, alas,
there is the well-known snag that if the key does not
exist in the table, you need to insert it yourself.
Which means you get a race condition that, as far as I
can see, ets can't handle safely. (Or could I be more
creative?)

6. More intuitive APIs -- it's been instructive, and a
bit alarming, to see how people outside the "core"
community have had to struggle with false starts on
this. More and better documentation and tutorials.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From lenartlad@REDACTED  Mon Oct 22 09:55:24 2007
From: lenartlad@REDACTED (Ladislav Lenart)
Date: Mon, 22 Oct 2007 09:55:24 +0200
Subject: [erlang-questions] Programming Erlang book code question
In-Reply-To: <4e3934860710211534p5e64f03cmdc269e331c973fcb@mail.gmail.com>
References: <4e3934860710211534p5e64f03cmdc269e331c973fcb@mail.gmail.com>
Message-ID: <471C576C.8050200@volny.cz>

Hello,

(some of) my answers to your questions are inlined below...

Aaron Feng wrote:
> The following code snippet appeared on page 352:
> 
> -module(sellaprime_supervisor).
> -behaviour(supervisor). % see erl -man supervisor
> -export([start/0, start_in_shell_for_testing/0, start_link/1, init/1]).
> 
> start() -> spawn(fun() ->
>                         supervisor:start_link({local,?MODULE}, ?MODULE, 
> _Arg = [])
>                         end).
> 
> start_in_shell_for_testing() ->
>   {ok, Pid} = supervisor:start_link({local,?MODULE}, ?MODULE, _Arg = []),
>   unlink(Pid).
> 
> %% rest of the code intentionally left out
> 
> Question 1:
> 
> What's the benefit to call spawn for supervisor:start_link in the 
> start() function?  Why not just call supervisor:start_link without the 
> spawn?  Doesn't supervisor:start_link already spawn and register the 
> process already?

I don't understand it either.

> Question 2:
> 
> What's the benefit for start_in_shell_for_testing() function?  Why not 
> just call start() function when testing in the shell?  I know what the 
> unlink(Pid) does, but why unlinking it?  What if unlink it left out?

When you cause an error in the shell (syntax or other), the shell is
killed (and restarted). However when the supervisor is started from
the shell, it will be linked to it and will be killed as well because
the supervisor thinks that its parent supervisor (the shell) exited
and is obliged to do the same.

> Question 3:
> 
> I don't understand _Arg = [] ?  why not just pass [] as the last 
> argument of start_link?

The only reason that comes to my mind is the readability of the code.
This way you know that an empty list is actually an argument list.

Hope this helps,

Ladislav Lenart




From dcaoyuan@REDACTED  Mon Oct 22 12:05:52 2007
From: dcaoyuan@REDACTED (Caoyuan)
Date: Mon, 22 Oct 2007 18:05:52 +0800
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <643517.43588.qm@web38804.mail.mud.yahoo.com>
References: <65b2728e0710211312g786909f6l8c2e6fd7760015d@mail.gmail.com>
	<643517.43588.qm@web38804.mail.mud.yahoo.com>
Message-ID: <ca96bd270710220305p5d239afdtd5dcc4fc34dc6bdc@mail.gmail.com>

It seems GC is also a key for performance now :-) per my tbray4.erl
code, which uses plain Dict

After I added [{min_heap_size, 3000}] to spawn_opt, the elapsed time
dropped from 7.7 sec to 5.7s immediately.

The code is at:
http://blogtrader.net/page/dcaoyuan/entry/learning_coding_parallelization_was_tim


On 10/22/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> > Anders sent me his code and I ran it on my 8-core
> > Linux box, with the
> > following performance results. VICTORY is right! :-)
> >
> > real    0m1.904s
> > user    0m7.917s
> > sys     0m1.185s
> >
> > Like I mentioned to Anders in private email, it's
> > nice to have someone more
> > experienced with Erlang finally taking a look at
> > this; I'm still a relative
> > newbie.
> > One thing I've liked about this entire exercise is
> > that the early attempts
> > at solving Tim Bray's Wide Finder in Erlang were
> > taking minutes to execute
> > and were providing only partial answers. Several of
> > us then started
> > whittling away at it, and because of the richness of
> > the language, we had a
> > variety of different avenues to explore. Over time,
> > we've vastly increased
> > the performance of our solutions. Anders's solution
> > now beats Ruby on the
> > same machine by about 0.3s, and because of the way
> > it uses multiple cores,
> > it will likely execute extremely quickly and
> > efficiently when Tim gets a
> > chance to try it on his T5120.
> >
> > Yes, fast solutions in other languages were quickly
> > found, but those had
> > almost nowhere to go beyond their initial forms in
> > terms of improvement, not
> > because they were already so fast, but because the
> > languages ran out of
> > alternatives. This is especially true when it comes
> > to taking advantage of
> > the T5120's many cores. I'm a fan of many languages,
> > including Ruby, Python,
> > Perl, and C++, all of which have figured prominently
> > in the collection of
> > various Wide Finder solutions. But for my money,
> > Erlang has fulfilled Tim's
> > original wishes the best, which is to take the best
> > possible advantage of
> > all those cores.
>
> Well done, everyone. You've chewed this over pretty
> well, I'd say, and it has been interesting to see how
> things have improved over time. Here are a couple of
> thoughts on further improvements:
>
> 1. Native code compilation? It's a bit hit-and-miss,
> but this could be the sort of problem that gains from
> it.
>
> 2. The speedup is 4.15 on 8 cores (if I'm reading
> things right: user/real). What is the bottleneck? Too
> small input, too much I/O, or is there something that
> could be improved or tuned further?
>
> And for language fans, the language itself could be a
> bit more helpful. While we haven't emphasized regexp
> crunching, it still seems like things could be easier.
> Here are some quick thoughts.
>
> 0. Not having to write the Boyer-Moore stuff by hand
>
> 1. Working with binaries like strings should be easier
>
> 2. Reading lines from files
>
> 3. Appropriate data chunking for file processing
> (people tried all from a few KB to several MB per
> chunk -- could the system figure out an appropriate
> size on its own?)
>
> 4. Perhaps a streaming interface would be even better?
> Jay Nelson suggested one a few years ago.
>
> 5. When looking at the use of dictionaries, it struck
> me that ets:update_counter/3 could have avoided the
> use of dictionaries and merging altogether. But, alas,
> there is the well-known snag that if the key does not
> exist in the table, you need to insert it yourself.
> Which means you get a race condition that, as far as I
> can see, ets can't handle safely. (Or could I be more
> creative?)
>
> 6. More intuitive APIs -- it's been instructive, and a
> bit alarming, to see how people outside the "core"
> community have had to struggle with false starts on
> this. More and better documentation and tutorials.
>
> Best,
> Thomas
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
- Caoyuan


From tonyg@REDACTED  Mon Oct 22 13:46:05 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Mon, 22 Oct 2007 12:46:05 +0100
Subject: [erlang-questions] Stability of "-hybrid"
Message-ID: <471C8D7D.4050700@lshift.net>

Hi all,

Is the "-hybrid" option to erl considered stable?

I only ask because it seems relatively prone to SIGSEGV, compared to
running without the flag :-)

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg@REDACTED


From thomasl_erlang@REDACTED  Mon Oct 22 14:11:56 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 22 Oct 2007 05:11:56 -0700 (PDT)
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <ca96bd270710220305p5d239afdtd5dcc4fc34dc6bdc@mail.gmail.com>
Message-ID: <467328.34572.qm@web38804.mail.mud.yahoo.com>


--- Caoyuan <dcaoyuan@REDACTED> wrote:

> It seems GC is also a key for performance now :-)
> per my tbray4.erl
> code, which uses plain Dict
> 
> After I added [{min_heap_size, 3000}] to spawn_opt,
> the elapsed time
> dropped from 7.7 sec to 5.7s immediately.
> 
> The code is at:
>
http://blogtrader.net/page/dcaoyuan/entry/learning_coding_parallelization_was_tim

That's a 35% speedup just from sizing the heap right.
So it seems the internal VM heuristics for this could
be tuned or improved.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From alexander.lamb@REDACTED  Mon Oct 22 14:47:44 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Mon, 22 Oct 2007 14:47:44 +0200
Subject: [erlang-questions] Design methodology going from Object oriented to
	functional programming?
Message-ID: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>

Hello list,

I am trying to understand what is the design process (intellectual,  
that is) when building a program in Erlang.

Indeed, in the object oriented world, I would start by finding what  
my classes might be and the relationship between them. Gradually I  
would add functions (class or instance methods) to the classes in  
order to provide solid foundations on top of which I can write an  
application.

For example, I could have PERSON, PROFILE, ROLE, FEATURE, etc... and  
decide a PROFILE is a collection of FEATURES. A PERSON can have 0 or  
many ROLES. A ROLE is a PROFILE on a given area (a department for  
example) for a given time. I would then add functions such as "give  
all the active roles for the user" or "what features give that  
profile" or "does the user have a given feature for that department".
I admit it is more complexe than that, but you get the idea.

Obviously, this doesn't seem to be the way to go with Erlang.  
Intuitively, I would start making a list of all the functions which  
will allow me to interract with my application. In that case I could  
have "give me all users with an active role on that department",  
etc... Then by implementing those high level functions I would split  
them into pieces by calling smaller simpler functions. The underlying  
data structure will "just follow" or "appear" naturally.

Hence: object oriented design is "data structure and relationships  
first, functions second" and functional design is "functions first,  
data structure second".

Am I being over simplistic here. Are there some guidelines as to how  
one can approach a problem when creating a new program? Especially  
programs which deal with persistent data, not protocole analysers or  
socket servers!

Thanks,

Alex
--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/e7dd1c31/attachment.htm>

From joelr1@REDACTED  Mon Oct 22 14:23:09 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Mon, 22 Oct 2007 13:23:09 +0100
Subject: [erlang-questions] Yecc and grammar ambiguities
Message-ID: <DCE97498-0C2F-45C6-9C0D-6506D72CF838@gmail.com>

How does Yecc handle grammar ambiguities?

Is there any debugging support to figure out what's getting shifted,  
reduced, etc?

	Thanks, Joel

--
http://wagerlabs.com







From kostis@REDACTED  Mon Oct 22 15:02:23 2007
From: kostis@REDACTED (Kostis Sagonas)
Date: Mon, 22 Oct 2007 16:02:23 +0300
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <467328.34572.qm@web38804.mail.mud.yahoo.com>
References: <467328.34572.qm@web38804.mail.mud.yahoo.com>
Message-ID: <471C9F5F.1030305@cs.ntua.gr>

Thomas Lindgren wrote:
> 
> That's a 35% speedup just from sizing the heap right.
> So it seems the internal VM heuristics for this could
> be tuned or improved.

Yes, of course. The question is exactly how to satisfy
all needs, especially since they are often contradictory.

The `process-local heap' architecture of the Erlang VM
has made certain choices: stating with a very small heap
(233 words) just so that many thousands (millions?) of
processes can be serviced without running the risk of
crashing the system due to lack of memory.

Of course, if only say 10 processes are ever going to run in
the VM (which I suspect is the case for most micro-benchmarks,
or for the average user out there) this number is ridiculously
low.

In most of today's machines, I strongly suspect that this
233 number can be increased to at least by a factor 10 -- and
perhaps it should -- but the OTP folks should first carefully
examine possible consequences before making this decision. (*)

Ideally, these consequences should be measured on real
applications, not on (toy) micro-benchmarks.  Having such a
repository of real applications that can be used as benchmarks
is something that is currently lacking in the Erlang community.
Contributions welcome.

Kostis

(*) The HiPE group can help in this -- this is a good MS Thesis.


From anders.nygren@REDACTED  Mon Oct 22 15:00:36 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 22 Oct 2007 08:00:36 -0500
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <643517.43588.qm@web38804.mail.mud.yahoo.com>
References: <65b2728e0710211312g786909f6l8c2e6fd7760015d@mail.gmail.com>
	<643517.43588.qm@web38804.mail.mud.yahoo.com>
Message-ID: <d3520a2f0710220600y75b2a3d5n593e3e2154f2e470@mail.gmail.com>

On 10/22/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> > Anders sent me his code and I ran it on my 8-core
> > Linux box, with the
> > following performance results. VICTORY is right! :-)
> >
> > real    0m1.904s
> > user    0m7.917s
> > sys     0m1.185s
> >
> > Like I mentioned to Anders in private email, it's
> > nice to have someone more
> > experienced with Erlang finally taking a look at
> > this; I'm still a relative
> > newbie.
> > One thing I've liked about this entire exercise is
> > that the early attempts
> > at solving Tim Bray's Wide Finder in Erlang were
> > taking minutes to execute
> > and were providing only partial answers. Several of
> > us then started
> > whittling away at it, and because of the richness of
> > the language, we had a
> > variety of different avenues to explore. Over time,
> > we've vastly increased
> > the performance of our solutions. Anders's solution
> > now beats Ruby on the
> > same machine by about 0.3s, and because of the way
> > it uses multiple cores,
> > it will likely execute extremely quickly and
> > efficiently when Tim gets a
> > chance to try it on his T5120.
> >
> > Yes, fast solutions in other languages were quickly
> > found, but those had
> > almost nowhere to go beyond their initial forms in
> > terms of improvement, not
> > because they were already so fast, but because the
> > languages ran out of
> > alternatives. This is especially true when it comes
> > to taking advantage of
> > the T5120's many cores. I'm a fan of many languages,
> > including Ruby, Python,
> > Perl, and C++, all of which have figured prominently
> > in the collection of
> > various Wide Finder solutions. But for my money,
> > Erlang has fulfilled Tim's
> > original wishes the best, which is to take the best
> > possible advantage of
> > all those cores.
>
> Well done, everyone. You've chewed this over pretty
> well, I'd say, and it has been interesting to see how
> things have improved over time. Here are a couple of
> thoughts on further improvements:
>
> 1. Native code compilation? It's a bit hit-and-miss,
> but this could be the sort of problem that gains from
> it.
>
> 2. The speedup is 4.15 on 8 cores (if I'm reading
> things right: user/real). What is the bottleneck? Too
> small input, too much I/O, or is there something that
> could be improved or tuned further?
>
> And for language fans, the language itself could be a
> bit more helpful. While we haven't emphasized regexp
> crunching, it still seems like things could be easier.
> Here are some quick thoughts.
>
> 0. Not having to write the Boyer-Moore stuff by hand
>
> 1. Working with binaries like strings should be easier
>
> 2. Reading lines from files
>
> 3. Appropriate data chunking for file processing
> (people tried all from a few KB to several MB per
> chunk -- could the system figure out an appropriate
> size on its own?)
>
> 4. Perhaps a streaming interface would be even better?
> Jay Nelson suggested one a few years ago.
>
> 5. When looking at the use of dictionaries, it struck
> me that ets:update_counter/3 could have avoided the
> use of dictionaries and merging altogether. But, alas,
> there is the well-known snag that if the key does not
> exist in the table, you need to insert it yourself.
> Which means you get a race condition that, as far as I
> can see, ets can't handle safely. (Or could I be more
> creative?)
>
> 6. More intuitive APIs -- it's been instructive, and a
> bit alarming, to see how people outside the "core"
> community have had to struggle with false starts on
> this. More and better documentation and tutorials.
>
> Best,
> Thomas
>

I did the change to ets:update_counter last night, on my dual
core laptop that made an improvement from my previously
reported
real    0m6.305s
user    0m10.149s
sys     0m0.380s

to
real    0m5.094s
user    0m8.781s
sys     0m0.352s

To avoid the update_counter race condition I have only one process
that all workers report their matches to.

I had some problems with native compilation but finally made it work
and
real    0m2.192s
user    0m3.260s
sys     0m0.336s

/Anders


From kenneth.lundin@REDACTED  Mon Oct 22 15:17:25 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Mon, 22 Oct 2007 15:17:25 +0200
Subject: [erlang-questions] Stability of "-hybrid"
In-Reply-To: <471C8D7D.4050700@lshift.net>
References: <471C8D7D.4050700@lshift.net>
Message-ID: <de422a40710220617m679a85efo91bbcbf6364281c1@mail.gmail.com>

The hybrid option should be regarded as experimental. And we don't test in R11B.

It will be regarded as experimental in the upcoming R12B release as well.

/Regards Kenneth (Erlang/OTP team at Ericsson)


On 10/22/07, Tony Garnock-Jones <tonyg@REDACTED> wrote:
> Hi all,
>
> Is the "-hybrid" option to erl considered stable?
>
> I only ask because it seems relatively prone to SIGSEGV, compared to
> running without the flag :-)
>
> Regards,
>  Tony
> --
>  [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
>   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
>  []  [] http://www.lshift.net/ | Email: tonyg@REDACTED
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From vances@REDACTED  Mon Oct 22 16:07:14 2007
From: vances@REDACTED (Vance Shipley)
Date: Mon, 22 Oct 2007 10:07:14 -0400
Subject: [erlang-questions] Design methodology going from
	Object	oriented to functional programming?
In-Reply-To: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
References: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
Message-ID: <20071022140643.GA592@little-black-book.motivity.ca>

Andrew,

As a Concurrency Oriented Programming Language (COPL) the first
thing I believe you want to do is to decide what processes you
will use.  After modeling the concurrency aspects the next thing
I do is to model the supervision heirarchy.  Only after this is
done do I reason about what a user API will look like.

	-Vance

On Mon, Oct 22, 2007 at 02:47:44PM +0200, Alexander Lamb wrote:
}   Indeed, in the object oriented world, I would start by finding what my 
}   classes might be and the relationship between them. Gradually I would add 
}   functions (class or instance methods) to the classes in order to provide 
}   solid foundations on top of which I can write an application.


From alexander.lamb@REDACTED  Mon Oct 22 16:19:00 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Mon, 22 Oct 2007 16:19:00 +0200
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <20071022140643.GA592@little-black-book.motivity.ca>
References: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
	<20071022140643.GA592@little-black-book.motivity.ca>
Message-ID: <ACB2FA13-669A-4488-ADFD-EAA8C69FF0F3@rodanotech.ch>

Thanks,

I had obviously totally overlooked the processes aspect!

So, for example in a session-full web application, the initial  
logical step would be to define one process per connected user and  
see if in addition to that some other services could be made  
concurrent by adding processes. The supervision tree is less clear  
for me, especially in a web application. I suppose if you distribute  
processes on several nodes, you want to make sure the connected user  
process is notified in case of a problem and the faulty process gets  
restarted somewhere.

Related to this, I will also have some questions about fault  
tolerance, but this will be the subject of another mail.

Alex

Le 22 oct. 07 ? 16:07, Vance Shipley a ?crit :

> Andrew,
>
> As a Concurrency Oriented Programming Language (COPL) the first
> thing I believe you want to do is to decide what processes you
> will use.  After modeling the concurrency aspects the next thing
> I do is to model the supervision heirarchy.  Only after this is
> done do I reason about what a user API will look like.
>
> 	-Vance
>
> On Mon, Oct 22, 2007 at 02:47:44PM +0200, Alexander Lamb wrote:
> }   Indeed, in the object oriented world, I would start by finding  
> what my
> }   classes might be and the relationship between them. Gradually I  
> would add
> }   functions (class or instance methods) to the classes in order  
> to provide
> }   solid foundations on top of which I can write an application.
>

--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/12c823d8/attachment.htm>

From jay@REDACTED  Mon Oct 22 16:46:16 2007
From: jay@REDACTED (Jay Nelson)
Date: Mon, 22 Oct 2007 07:46:16 -0700
Subject: [erlang-questions] Design methodology going from
	Object	oriented to functional programming?
Message-ID: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>

I wrote a paper for the 2004 ICFP on designing structured systems  
using processes.  You can find it on my website at http:// 
www.duomark.com/erlang/index.html

jay



From jay@REDACTED  Mon Oct 22 17:01:56 2007
From: jay@REDACTED (Jay Nelson)
Date: Mon, 22 Oct 2007 08:01:56 -0700
Subject: [erlang-questions] Strange optimization result
Message-ID: <F673749D-8859-4CE6-9E11-3910DCB17101@duomark.com>

Thomas Lindgren wrote:

 > 4. Perhaps a streaming interface would be even better?
 > Jay Nelson suggested one a few years ago.

I've been following this thread here and in the blogosphere, but I am  
very busy with some code I am working on and am trying valiantly to  
wait until I am free before giving an analysis.  It will take some  
time to discuss all the issues.

For those who haven't read the streaming paper it is on my website at  
http://www.duomark.com/erlang/index.html and describes how to build  
BIFs that can speed up binary handling.  That work was an initial  
study of whether binary handling could be accelerated by including a  
low-level library of binary functions.  It gives performance  
comparisons on an old VM for binary matching versus BIFs.  This  
problem is very amenable to some of the techniques presented in the  
paper -- in fact, similar types of problems were what I had in mind  
when I started the investigation.

I did further work related to the parsing of binaries over the last  
year, but it is very incomplete.  It will be 6 months or so (based on  
my past track record rather than my optimism) before I will have a  
more complete package and analysis of performance.  I expect to use  
Tim's problem as a motivation for the work.

The main issues I see are as follows:

1) erlang has lots of options, so the developers were able to try  
several approaches
2) The performance issues are non-obvious without experience
3) There has been a lot of code slinging and micro-tweaking with good  
progress, but not the real analysis of the problem that would  
identify the architectural underpinnings for a good solution  
(although the participants now have some good experience with  
different erlang tools)
4) Doing a proper account of #3 as Thomas suggested would give an  
indication of some potential for improving the erlang out-of-the-box  
performance on data analysis applications
5) The solutions are large and complicated pieces of code to replace  
a 5-liner; erlang should be able to do something useful in 10-20  
lines of code
6) While the participants have tried to wring every ounce of  
performance out of their solutions, the prospect of a general  
solution that could easily be adapted to other problems is strong and  
should be faster than a ruby implementation on multi-core, although  
probably a little slower than the special purpose solutions being  
offered

jay



From thomasl_erlang@REDACTED  Mon Oct 22 17:17:22 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 22 Oct 2007 08:17:22 -0700 (PDT)
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <d3520a2f0710220600y75b2a3d5n593e3e2154f2e470@mail.gmail.com>
Message-ID: <272286.69016.qm@web38815.mail.mud.yahoo.com>


--- Anders Nygren <anders.nygren@REDACTED> wrote:

> I did the change to ets:update_counter last night,
> on my dual
> core laptop that made an improvement from my
> previously
> reported
> real    0m6.305s
> user    0m10.149s
> sys     0m0.380s
> 
> to
> real    0m5.094s
> user    0m8.781s
> sys     0m0.352s
> 
> To avoid the update_counter race condition I have
> only one process
> that all workers report their matches to.
> 
> I had some problems with native compilation but
> finally made it work
> and
> real    0m2.192s
> user    0m3.260s
> sys     0m0.336s

Nice -- so native compilation yields a 2.3x speedup
versus the ets-based version (and 2.87x versus the
original). Have you tried setting that process to
priority high? (Can be dangerous in general, but in
the interest of science ... :-)

Parallelization yields 1.78x speedup on the first
version and 1.49x on the native code version. That's
pretty good too on two cores.

(I think the ultimate limit in most any language would
be a time of about 0.33 seconds, the sys time. But at
this point, we are actually within a factor 10 of
that.)

Okay, I just thought of something marginally more
clever when using ets. Maybe you did it this way,
even.

All the matching processes do this when they find a
match 'M':

case catch ets:update_counter(Tab, M, 1) of
   {'EXIT', Rsn} -> match_table_owner ! {init_key, M};
   _ -> ok
end

and the match_table_owner (which will be the sole
process inserting new matches into the table) does
something like

receive
  {init_key, M} ->
    case catch ets:update_counter(Tab, M, 1) of
       {'EXIT', _} -> ets:insert(Tab, {M, 1});
       _ -> ok
    end, ...;
  ...
end

This detects and handles the race condition by
funneling all initializations through the
match_table_owner and making the init there
idempotent. Furthermore, post-init updates will be
done by each matching process without sending
anything. Sounds potentially promising.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From alexander.lamb@REDACTED  Mon Oct 22 17:21:53 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Mon, 22 Oct 2007 17:21:53 +0200
Subject: [erlang-questions] Design methodology going from
	Object	oriented to functional programming?
In-Reply-To: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
References: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
Message-ID: <9DD7021A-7659-48D0-8CCC-70B67F609BB9@rodanotech.ch>

Thanks a lot!!

Everyday I discover new web sites with very valuable content for  
Erlang... Even coming from the ADA then Objective-C world it is hard  
to change the way one thinks about software development (especially  
after years of Java).

Alex

Le 22 oct. 07 ? 16:46, Jay Nelson a ?crit :

> I wrote a paper for the 2004 ICFP on designing structured systems
> using processes.  You can find it on my website at http://
> www.duomark.com/erlang/index.html
>
> jay
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/04eb3196/attachment.htm>

From aaron.feng@REDACTED  Mon Oct 22 17:25:48 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Mon, 22 Oct 2007 11:25:48 -0400
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
References: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
Message-ID: <4e3934860710220825m58662251l4515726b34aec56e@mail.gmail.com>

I also came from OOP background, and attempting to translate some of the
concepts exists in the domain into processes and data classes as tuples for
my side project.  So far, dividing up the responsibility into processes
seems be working out fine.  I would be interested to see what others think
about this approach.

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/ff54c971/attachment.htm>

From thomasl_erlang@REDACTED  Mon Oct 22 17:37:26 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 22 Oct 2007 08:37:26 -0700 (PDT)
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <471C9F5F.1030305@cs.ntua.gr>
Message-ID: <145698.312.qm@web38814.mail.mud.yahoo.com>


--- Kostis Sagonas <kostis@REDACTED> wrote:

> Thomas Lindgren wrote:
> > 
> > That's a 35% speedup just from sizing the heap
> right.
> > So it seems the internal VM heuristics for this
> could
> > be tuned or improved.
> 
> Yes, of course. The question is exactly how to
> satisfy
> all needs, especially since they are often
> contradictory. /.../

I think an online approach could be interesting here.
For example, record memory usage and GC
characteristics of processes spawned at one site (or
otherwise categorized somehow) and use that, along
with current memory use, to make educated (but
non-binding) guesses.

> Having such a
> repository of real applications that can be used as
> benchmarks
> is something that is currently lacking in the Erlang
> community.
> Contributions welcome.

The drawback of 'real code' is that it is often
proprietary and relies on environments that are
tedious to set up. It can be difficult to analyze and
optimize too, as readers looking in the Hipe archives
can see for themselves :-)

A few years ago, I collected and used a benchmark
suite of medium-size programs, based on OTP. These
exercised the compiler, mnesia, ASN1 (encode/decode
LDAPv2) and gen_tcp.

The advantage was that the code was free and
optimizing these apps would help everyone. A potential
disadvantage was that new versions appear regularly,
so the benchmark suite would have to evolve
appropriately and comparison between versions would be
iffy.

Alas, it didn't catch on at the time. But maybe the
right time has come. Today, I'd suggest adding yaws
and maybe YXA, the wide finder (of course!), and
suchlike to the mix.

> (*) The HiPE group can help in this -- this is a
> good MS Thesis.

I think so too.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From anders.nygren@REDACTED  Mon Oct 22 18:02:51 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 22 Oct 2007 11:02:51 -0500
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <272286.69016.qm@web38815.mail.mud.yahoo.com>
References: <d3520a2f0710220600y75b2a3d5n593e3e2154f2e470@mail.gmail.com>
	<272286.69016.qm@web38815.mail.mud.yahoo.com>
Message-ID: <d3520a2f0710220902ja2404bavb6c596e8d959ea8f@mail.gmail.com>

On 10/22/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Anders Nygren <anders.nygren@REDACTED> wrote:
>
> > I did the change to ets:update_counter last night,
> > on my dual
> > core laptop that made an improvement from my
> > previously
> > reported
> > real    0m6.305s
> > user    0m10.149s
> > sys     0m0.380s
> >
> > to
> > real    0m5.094s
> > user    0m8.781s
> > sys     0m0.352s
> >
> > To avoid the update_counter race condition I have
> > only one process
> > that all workers report their matches to.
> >
> > I had some problems with native compilation but
> > finally made it work
> > and
> > real    0m2.192s
> > user    0m3.260s
> > sys     0m0.336s
>
> Nice -- so native compilation yields a 2.3x speedup
> versus the ets-based version (and 2.87x versus the
> original). Have you tried setting that process to
> priority high? (Can be dangerous in general, but in
> the interest of science ... :-)

I just tried it but it actually made things worse.

>
> Parallelization yields 1.78x speedup on the first
> version and 1.49x on the native code version. That's
> pretty good too on two cores.
>
> (I think the ultimate limit in most any language would
> be a time of about 0.33 seconds, the sys time. But at
> this point, we are actually within a factor 10 of
> that.)

Huh??
I thought that sys was the time spent in the kernel.

>
> Okay, I just thought of something marginally more
> clever when using ets. Maybe you did it this way,
> even.
>
> All the matching processes do this when they find a
> match 'M':
>
> case catch ets:update_counter(Tab, M, 1) of
>    {'EXIT', Rsn} -> match_table_owner ! {init_key, M};
>    _ -> ok
> end
>
> and the match_table_owner (which will be the sole
> process inserting new matches into the table) does
> something like
>
> receive
>   {init_key, M} ->
>     case catch ets:update_counter(Tab, M, 1) of
>        {'EXIT', _} -> ets:insert(Tab, {M, 1});
>        _ -> ok
>     end, ...;
>   ...
> end
>
> This detects and handles the race condition by
> funneling all initializations through the
> match_table_owner and making the init there
> idempotent. Furthermore, post-init updates will be
> done by each matching process without sending
> anything. Sounds potentially promising.

I have tested it but on two cores it makes no noticeable
difference, even though it sounds like a good idea.

/Anders


From dmercer@REDACTED  Mon Oct 22 18:23:10 2007
From: dmercer@REDACTED (David Mercer)
Date: Mon, 22 Oct 2007 11:23:10 -0500
Subject: [erlang-questions] Interfacing Erlang With IBM's Websphere MQ
Message-ID: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>

Anyone know if a module exists to interface an Erlang program with IBM's
Websphere MQ messaging queue?

 

Cheers,

 

David

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/7989b7a7/attachment.htm>

From thomasl_erlang@REDACTED  Mon Oct 22 18:26:51 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 22 Oct 2007 09:26:51 -0700 (PDT)
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <d3520a2f0710220902ja2404bavb6c596e8d959ea8f@mail.gmail.com>
Message-ID: <471501.36318.qm@web38804.mail.mud.yahoo.com>


--- Anders Nygren <anders.nygren@REDACTED> wrote:

> On 10/22/07, Thomas Lindgren
> <thomasl_erlang@REDACTED> wrote:
> >
> > (I think the ultimate limit in most any language
> would
> > be a time of about 0.33 seconds, the sys time. But
> at
> > this point, we are actually within a factor 10 of
> > that.)
> 
> Huh??
> I thought that sys was the time spent in the kernel.

I was thinking of a variation on Amdahl's Law: even
with infinite speedup of the application, you'd still
be stuck with running stuff in the kernel, which
limits potential speedup. For example, even if all
matching work could be done while waiting for more
data to arrive, the task would still need at least
0.33 seconds. But this is basically just a theoretical
rule of thumb.

> I have tested it but on two cores [using local
ets:update_counter/3] makes no
> noticeable
> difference, even though it sounds like a good idea.

Too bad. Well, good luck with it all.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From patrickdlogan@REDACTED  Mon Oct 22 20:23:52 2007
From: patrickdlogan@REDACTED (Patrick Logan)
Date: Mon, 22 Oct 2007 11:23:52 -0700
Subject: [erlang-questions] Interfacing Erlang With IBM's Websphere MQ
In-Reply-To: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>
References: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>
Message-ID: <e9447a400710221123lf324babod94b4209ac67f797@mail.gmail.com>

> Anyone know if a module exists to interface an Erlang program with IBM's
> Websphere MQ messaging queue?

Websphere MQ has a simple HTTP interface which may be a reasonable
place to begin...

* GET the current item on a queue
* DELETE and return the current item on a queue
* POST an item to a queue

Not truly "REST" but it is simple and lightweight.

Heavier mechanisms for Erlang would be one of the several Java
interfaces (JMS or MQ-specific), then go through Erlang's Java
library.

Or the MQ C/C++ interfaces.

-Patrick


From ciukes2@REDACTED  Mon Oct 22 21:25:52 2007
From: ciukes2@REDACTED (Marcin Maciukiewicz)
Date: Mon, 22 Oct 2007 20:25:52 +0100
Subject: [erlang-questions] Design methodology going from Object
 oriented to functional programming?
In-Reply-To: <9DD7021A-7659-48D0-8CCC-70B67F609BB9@rodanotech.ch>
References: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
	<9DD7021A-7659-48D0-8CCC-70B67F609BB9@rodanotech.ch>
Message-ID: <20071022202552.0a4804d8@mmaciukiewicz>


Hi Everyone,
I'm an erlang newbie with lots of OOP experience. 
I did my first hello-world apps, 
now I joined this list to learn more :)

On Mon, 22 Oct 2007 17:21:53 +0200
Alexander Lamb <alexander.lamb@REDACTED> wrote:
> Everyday I discover new web sites with very valuable content for  
> Erlang... 
Hope you find my links list useful:
http://www.diigo.com/user/ciukes/erlang

Share your list, please :)

Regards,
Marcin.


From rvirding@REDACTED  Mon Oct 22 23:00:30 2007
From: rvirding@REDACTED (Robert Virding)
Date: Mon, 22 Oct 2007 23:00:30 +0200
Subject: [erlang-questions] Posregex for windows
Message-ID: <3dbc6d1c0710221400r4437e2eq6858092a38d3aa0c@mail.gmail.com>

Hi,

I want to do some testing with a POSIX regex implementation but I'm stuck on
a windows machine with no C compiler. :-( Does anyone know where I can find
Henry Spencer's posregex library for windows? Or any other POSIX compliant
regex package to which Erlang has an interface? Already compiled interface
that is. :-( :-(

Thanks,
Robert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/60243014/attachment.htm>

From rvirding@REDACTED  Mon Oct 22 23:13:57 2007
From: rvirding@REDACTED (Robert Virding)
Date: Mon, 22 Oct 2007 23:13:57 +0200
Subject: [erlang-questions] Strange optimization result
In-Reply-To: <ca96bd270710220305p5d239afdtd5dcc4fc34dc6bdc@mail.gmail.com>
References: <65b2728e0710211312g786909f6l8c2e6fd7760015d@mail.gmail.com>
	<643517.43588.qm@web38804.mail.mud.yahoo.com>
	<ca96bd270710220305p5d239afdtd5dcc4fc34dc6bdc@mail.gmail.com>
Message-ID: <3dbc6d1c0710221413t4f531a05x2719505fc4a462a3@mail.gmail.com>

Actually it is a surprise to me that the dict version compares so favourably
to both the tuple version and the ets version considering that it is
implemented wholly in Erlang while the others are implemented in C in the
emulator. It must have got something right. :-)

It might help a lot if there was support for it in the emulator. Not
completely implement it there but have some support functions. There are
some parts which would be difficult to do in the emulator. Future EEP?

Robert

On 22/10/2007, Caoyuan <dcaoyuan@REDACTED> wrote:
>
> It seems GC is also a key for performance now :-) per my tbray4.erl
> code, which uses plain Dict
>
> After I added [{min_heap_size, 3000}] to spawn_opt, the elapsed time
> dropped from 7.7 sec to 5.7s immediately.
>
> The code is at:
>
> http://blogtrader.net/page/dcaoyuan/entry/learning_coding_parallelization_was_tim
>
>
> On 10/22/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
> >
> > --- Steve Vinoski <vinoski@REDACTED> wrote:
> >
> > > Anders sent me his code and I ran it on my 8-core
> > > Linux box, with the
> > > following performance results. VICTORY is right! :-)
> > >
> > > real    0m1.904s
> > > user    0m7.917s
> > > sys     0m1.185s
> > >
> > > Like I mentioned to Anders in private email, it's
> > > nice to have someone more
> > > experienced with Erlang finally taking a look at
> > > this; I'm still a relative
> > > newbie.
> > > One thing I've liked about this entire exercise is
> > > that the early attempts
> > > at solving Tim Bray's Wide Finder in Erlang were
> > > taking minutes to execute
> > > and were providing only partial answers. Several of
> > > us then started
> > > whittling away at it, and because of the richness of
> > > the language, we had a
> > > variety of different avenues to explore. Over time,
> > > we've vastly increased
> > > the performance of our solutions. Anders's solution
> > > now beats Ruby on the
> > > same machine by about 0.3s, and because of the way
> > > it uses multiple cores,
> > > it will likely execute extremely quickly and
> > > efficiently when Tim gets a
> > > chance to try it on his T5120.
> > >
> > > Yes, fast solutions in other languages were quickly
> > > found, but those had
> > > almost nowhere to go beyond their initial forms in
> > > terms of improvement, not
> > > because they were already so fast, but because the
> > > languages ran out of
> > > alternatives. This is especially true when it comes
> > > to taking advantage of
> > > the T5120's many cores. I'm a fan of many languages,
> > > including Ruby, Python,
> > > Perl, and C++, all of which have figured prominently
> > > in the collection of
> > > various Wide Finder solutions. But for my money,
> > > Erlang has fulfilled Tim's
> > > original wishes the best, which is to take the best
> > > possible advantage of
> > > all those cores.
> >
> > Well done, everyone. You've chewed this over pretty
> > well, I'd say, and it has been interesting to see how
> > things have improved over time. Here are a couple of
> > thoughts on further improvements:
> >
> > 1. Native code compilation? It's a bit hit-and-miss,
> > but this could be the sort of problem that gains from
> > it.
> >
> > 2. The speedup is 4.15 on 8 cores (if I'm reading
> > things right: user/real). What is the bottleneck? Too
> > small input, too much I/O, or is there something that
> > could be improved or tuned further?
> >
> > And for language fans, the language itself could be a
> > bit more helpful. While we haven't emphasized regexp
> > crunching, it still seems like things could be easier.
> > Here are some quick thoughts.
> >
> > 0. Not having to write the Boyer-Moore stuff by hand
> >
> > 1. Working with binaries like strings should be easier
> >
> > 2. Reading lines from files
> >
> > 3. Appropriate data chunking for file processing
> > (people tried all from a few KB to several MB per
> > chunk -- could the system figure out an appropriate
> > size on its own?)
> >
> > 4. Perhaps a streaming interface would be even better?
> > Jay Nelson suggested one a few years ago.
> >
> > 5. When looking at the use of dictionaries, it struck
> > me that ets:update_counter/3 could have avoided the
> > use of dictionaries and merging altogether. But, alas,
> > there is the well-known snag that if the key does not
> > exist in the table, you need to insert it yourself.
> > Which means you get a race condition that, as far as I
> > can see, ets can't handle safely. (Or could I be more
> > creative?)
> >
> > 6. More intuitive APIs -- it's been instructive, and a
> > bit alarming, to see how people outside the "core"
> > community have had to struggle with false starts on
> > this. More and better documentation and tutorials.
> >
> > Best,
> > Thomas
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam protection around
> > http://mail.yahoo.com
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> --
> - Caoyuan
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/12488c82/attachment.htm>

From saleyn@REDACTED  Tue Oct 23 03:39:24 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Mon, 22 Oct 2007 20:39:24 -0500
Subject: [erlang-questions] HiPE compilation option
Message-ID: <471D50CC.1030601@gmail.com>

One of my colleagues asked me how to enable HiPE compilation.  After 
explaining the +native command-line flag I went back to documentation 
and haven't found any reference to it in erlc, c, compile.  I think it 
must be an oversight as HiPE is an important feature that deserves to be 
documented.

Serge


From ok@REDACTED  Tue Oct 23 04:08:07 2007
From: ok@REDACTED (Richard A. O'Keefe)
Date: Tue, 23 Oct 2007 15:08:07 +1300
Subject: [erlang-questions] Fwd: Why single-assignment with non-shared
	state?
In-Reply-To: <6A31E2B2-FD58-43B9-A5C8-D0A90C57094D@gmail.com>
References: <54588F09-37BD-4136-BA7C-CD2433EBAC7B@gmail.com>
	<6A31E2B2-FD58-43B9-A5C8-D0A90C57094D@gmail.com>
Message-ID: <E787BEFE-6AB4-4727-9ADC-AB3A014C057E@cs.otago.ac.nz>

On 22 Oct 2007, at 2:05 am, Tobias Gerdin wrote:

> Well as long as the expressions you match on doesn't change during
> matching I do not see the problem. There are several prominent
> languages such as the ML-family, Scala, or even Lisps which feature
> both pattern matching and assignment.

I know nothing about Scala.

However, I keep both SML and Mlton on my machines (hmm, I think I may
have MoscowML somewhere as well).  SML DOES NOT ALLOW PATTERN MATCHING
ON MUTABLE DATA.  SML's mutable data are "references" (boxes holding a
single variable) and "arrays".  You cannot look inside either of these
using pattern matching.

The only "pattern matching" feature standardly available in Lisp is
destructuring-bind (and its close relatives).  It matches mutable
cons cells because Lisp doesn't have any other kind.  Reading any
Scheme or Lisp tutorial, with its dire warnings against changing
lists that happened to appear as literal data, may provide some idea
of the trouble mutable data can get you into.



From aaron.feng@REDACTED  Tue Oct 23 04:29:51 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Mon, 22 Oct 2007 22:29:51 -0400
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
References: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
Message-ID: <4e3934860710221929y5b0a775bm10a7df93f206003b@mail.gmail.com>

Hi Jay,

I found the paper very insightful.  I understand the space constraint that
some of the code sample has to be removed.  Is the complete code sample
available for download?

Aaron

On 10/22/07, Jay Nelson <jay@REDACTED> wrote:
>
> I wrote a paper for the 2004 ICFP on designing structured systems
> using processes.  You can find it on my website at http://
> www.duomark.com/erlang/index.html
>
> jay
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/7b3bfb0a/attachment.htm>

From yinso.chen@REDACTED  Tue Oct 23 04:41:26 2007
From: yinso.chen@REDACTED (YC)
Date: Mon, 22 Oct 2007 19:41:26 -0700
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
References: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
Message-ID: <779bf2730710221941y4865b02bo5c842e94ced49c96@mail.gmail.com>

Actually, at the level you are describing, there should be no difference
between FP and OOP.

While OOP definitely emphasize data and relations, it is not the only
paradigm that does so - and given your background you should be experienced
with the relational paradigm, which has even heavier emphasis on data and
relations, but is not OOP.

To leverage your knowledge on relations - you can pretend sql queries are
functions, i.e. select is a function, update is a function, and insert is a
function.  Then instead of writing

object.insert()

you write

insert (object) - which looks a lot like the sql query insert into table ...


That's it.  In the Java style OOP the head of the statement is an object,
and in FP the head of the statement is a function name.  But in either case
you need to model the same world.

An extremely crude way of thinking about FP is that it decouples the data
from the function (or vice versa, that OOP couples functions and data).
That means some facilities that you've taken for granted in OOP, such as
inheritance, polymorphism, etc., will no longer be available.  But FP have a
different approaches to address these problems, and that's where the rubber
meet the road.

Erlang's process model is basically the Actor model - the idea of everything
is an actor feels similar to everything is an object in OOP.  So as others
have alluded to instead of thinking in objects you can think in processes.
But Actor model is independent of FP or OOP, so you would still have to get
used to the FP part in Erlang.

You might want to check out http://www.math.chalmers.se/~rjmh/Papers/whyfp.html
<http://www.math.chalmers.se/%7Erjmh/Papers/whyfp.html>.  A higher level
introduction is http://www.defmacro.org/ramblings/fp.html.

Cheers,
yc

On 10/22/07, Alexander Lamb < alexander.lamb@REDACTED > wrote:
>
> Hello list,
> I am trying to understand what is the design process (intellectual, that
> is) when building a program in Erlang.
>
> Indeed, in the object oriented world, I would start by finding what my
> classes might be and the relationship between them. Gradually I would add
> functions (class or instance methods) to the classes in order to provide
> solid foundations on top of which I can write an application.
>
> For example, I could have PERSON, PROFILE, ROLE, FEATURE, etc... and
> decide a PROFILE is a collection of FEATURES. A PERSON can have 0 or many
> ROLES. A ROLE is a PROFILE on a given area (a department for example) for a
> given time. I would then add functions such as "give all the active roles
> for the user" or "what features give that profile" or "does the user have a
> given feature for that department".
> I admit it is more complexe than that, but you get the idea.
>
> Obviously, this doesn't seem to be the way to go with Erlang. Intuitively,
> I would start making a list of all the functions which will allow me to
> interract with my application. In that case I could have "give me all users
> with an active role on that department", etc... Then by implementing those
> high level functions I would split them into pieces by calling smaller
> simpler functions. The underlying data structure will "just follow" or
> "appear" naturally.
>
> Hence: object oriented design is "data structure and relationships first,
> functions second" and functional design is "functions first, data structure
> second".
>
> Am I being over simplistic here. Are there some guidelines as to how one
> can approach a problem when creating a new program? Especially programs
> which deal with persistent data, not protocole analysers or socket servers!
>
> Thanks,
>
> Alex
>   --
> Alexander Lamb
> Founding Associate
> RODANOTECH S?rl
>
> 4 ch. de la Tour de Champel
> 1206 Geneva
> Switzerland
>
> Tel:  022 347 77 37
> Fax: 022 347 77 38
>
> http://www.rodanotech.ch
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071022/0fc13763/attachment.htm>

From jay@REDACTED  Tue Oct 23 06:51:44 2007
From: jay@REDACTED (Jay Nelson)
Date: Mon, 22 Oct 2007 21:51:44 -0700
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <4e3934860710221929y5b0a775bm10a7df93f206003b@mail.gmail.com>
References: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
	<4e3934860710221929y5b0a775bm10a7df93f206003b@mail.gmail.com>
Message-ID: <FC68D843-F1BE-4859-AE50-E966D81B4F82@duomark.com>

I haven't looked at for a couple years.  I don't have the code on the  
web.  I will have to look around and see if I can find it on another  
computer.

jay

On Oct 22, 2007, at 7:29 PM, Aaron Feng wrote:

> Hi Jay,
>
> I found the paper very insightful.  I understand the space  
> constraint that some of the code sample has to be removed.  Is the  
> complete code sample available for download?
>
> Aaron
>



From alexander.lamb@REDACTED  Tue Oct 23 12:12:11 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Tue, 23 Oct 2007 12:12:11 +0200
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
In-Reply-To: <FC68D843-F1BE-4859-AE50-E966D81B4F82@duomark.com>
References: <DA0EE898-E3AE-4B29-B4EE-99807696A752@duomark.com>
	<4e3934860710221929y5b0a775bm10a7df93f206003b@mail.gmail.com>
	<FC68D843-F1BE-4859-AE50-E966D81B4F82@duomark.com>
Message-ID: <DCA9528E-9C82-459F-A9E7-B0420C3D55EB@rodanotech.ch>

Thanks to all for those interesting pointers.

I read the article acm2004. I will have questions about splitting  
everything into processes.
I had lots of fun reading the articles from http://www.defmacro.org/ 
ramblings/articles.html

Alex

Le 23 oct. 07 ? 06:51, Jay Nelson a ?crit :

> I haven't looked at for a couple years.  I don't have the code on the
> web.  I will have to look around and see if I can find it on another
> computer.
>
> jay
>
> On Oct 22, 2007, at 7:29 PM, Aaron Feng wrote:
>
>> Hi Jay,
>>
>> I found the paper very insightful.  I understand the space
>> constraint that some of the code sample has to be removed.  Is the
>> complete code sample available for download?
>>
>> Aaron
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/4a64afd8/attachment.htm>

From exta7@REDACTED  Tue Oct 23 12:15:19 2007
From: exta7@REDACTED (Zvi)
Date: Tue, 23 Oct 2007 03:15:19 -0700 (PDT)
Subject: [erlang-questions]  http:request with delete method
Message-ID: <13361032.post@talk.nabble.com>


Anyone know, how to perform HTTP DELETE method with inets?

Is  this is a right way:

 {ok, Response} = http:request(delete, {URL,[]}, [], [{sync,true}]).

Also is there ibrowse with driver compiled for Windows? I tried to install
it via CEAN, but it installed without driver.

thanks in advance
Zvi
-- 
View this message in context: http://www.nabble.com/http%3Arequest-with-delete-method-tf4676439.html#a13361032
Sent from the Erlang Questions mailing list archive at Nabble.com.



From joelr1@REDACTED  Tue Oct 23 12:52:28 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 11:52:28 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
Message-ID: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>

I only have so much horizontal real-estate on my screen and I totally  
hate how xmlElement and xmlAttribute take so much of it. I also hate  
how XML parsing code looks in Erlang.

Am I the only one?

I think Simon Peyton-Jones (Haskell) said that the syntax is the user  
interface of a language. If so then Erlang has a user interface only  
a mother could love, at least when it comes to parsing XML!

	Joel

--
http://wagerlabs.com







From richardc@REDACTED  Tue Oct 23 13:21:20 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Tue, 23 Oct 2007 13:21:20 +0200
Subject: [erlang-questions] Design methodology going from Object
 oriented to	functional programming?
In-Reply-To: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
References: <E1EE4385-0637-4E57-AFEA-BB81235F2CB4@rodanotech.ch>
Message-ID: <471DD930.2010809@it.uu.se>

Alexander Lamb wrote:
> I am trying to understand what is the design process (intellectual, that 
> is) when building a program in Erlang.
> 
> Indeed, in the object oriented world, I would start by finding what my 
> classes might be and the relationship between them. Gradually I would 
> add functions (class or instance methods) to the classes in order to 
> provide solid foundations on top of which I can write an application.

In functional programming, you'd similarly begin with at least an idea of
what your data structures should be, that is, what information do you need
to work on, and what is the most suitable representation of that
information. A big difference from the OO thinking is that you'd immediately
try to identify and in as much as possible factor out all stateful data.
If you're lucky, all state can be mapped naturally to processes, and the
rest of the code is purely functional. Otherwise, you'll have to put some of
the state in a table or database. For many smaller problems, state is best
modelled as threading a State variable through those functions that need it.

I've seen very little written about programming methodology and design in
functional languages; perhaps this is because the field is too small (in $)
to attract the sort of people who write that sort of books. But it could
also have something to do with the following selected quotes:

   "It is better to have 100 functions operate on one data structure than
   10 functions on 10 data structures."
   (Alan Perlis)
   http://www.cs.yale.edu/quotes.html

   "I think part of the reason people in big companies like object-oriented
   programming is because it yields a lot of what looks like work. Something
   that might naturally be represented as, say, a list of integers, can now
   be represented as a class with all kinds of scaffolding and hustle and
   bustle."
   (Paul Graham on language design)
   http://www.paulgraham.com/langdes.html

   "... When I see patterns in my programs, I consider it a sign of trouble.
   The shape of a program should reflect only the problem it needs to solve.
   Any other regularity in the code is a sign, to me at least, that I'm using
   abstractions that aren't powerful enough..."
   (Paul Graham on the relative power of programming languages)
   http://www.paulgraham.com/icad.html

   "16 of 23 patterns are either invisible or simpler, ..."
   (Peter Norvig on Design Patterns in Dylan or Lisp)
   http://www.norvig.com/design-patterns/

<rant>I've read the GOF book on Design Patterns, and think it is a great
work. Every programmer should have read it, and keep an awareness of these
patterns in the back of his head. But to consciously use patterns, to the
point where you name classes after them and write in the documentation that
this-or-that implements the thingummy pattern, is just as misguided as if an
architect, after reading the original work by Christopher Alexander
(http://en.wikipedia.org/wiki/Christopher_Alexander) would keep noting in
his blueprints which sections of a building implement which architectural
patterns (or even name a back garden the "PUBLIC OUTDOOR ROOM").</rant>

     /Richard



From vladdu55@REDACTED  Tue Oct 23 13:47:59 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 23 Oct 2007 13:47:59 +0200
Subject: [erlang-questions] {eep} Metaprogramming in Erlang
Message-ID: <95be1d3b0710230447i3e6decf4se4fde0064aeb7005@mail.gmail.com>

Hello everybody,

I have been working on some ideas for quite a while and it didn't go
as fast as I would have liked, but now at least I have a draft that I
am happy with.

What I have in mind is (as the title says) adding metaprogramming
capabilities to Erlang.

The attached draft EEP is addressing some of the issues involved.
Since this offers a solution to some problems ventilated more or less
recently (last in row being Joel's XML rant :-), my belief is that it
might be an interesting topic to discuss.

The draft eep is inlined in this message (instead of being attached)
so that it will be easier to make comments next to the relevant part
of the document.

I welcome all input, so please don't feel shy!

best regards,
Vlad

---------------------------------------------------------------------------

EEP: XXXX
Title: Metaprogramming
Version: $Revision: $
Last-Modified: $Date: $
Author: Vlad Dumitrescu <vladdu55@REDACTED>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 23-Oct-2007
Erlang-Version: R12
Post-History:


Abstract
========

This EEP describes an extension to add elements of metaprogramming to
the language.  The basic ideas are those presented by Walid Taha et
al. in their work on multi-stage programming [1]_.  Code is given data type
status and operators for handling code are defined.

A prototype implementation is being worked on.

Background
==========

Metaprogramming is a powerful computing paradigm where programs and
data are interchangeable.  One can write programs that create and
manipulate other programs, that can then be run in order to obtain the
answer to one's problem.

This isn't the place to go into details about metaprogramming, there
is a lot to read in the literature and the Internet.  The source of
inspiration is the work of Walid Taha and team [1]_, please find more
information there.  Here we will focus on how to apply these
techniques in Erlang.


Motivation
==========

Being able to handle code as data has many use cases.  Some examples
with immediate benefits for current Erlang applications are:

Optimization by partial application
-----------------------------------

Computing a function of several parameters can be optimized by fixing
some of the parameters and generating a tailored function of the rest
of them. Since this generation is done offline, it may speed up
considerably the original call.

For example, the function ``pow(X, N)`` that raises x to the power of
n may be specialized for a given n. The generated function could look
like ::

  pow_27(X) ->
    P3 = X * X * X,
    P3 * P3 * P3.

which is faster than a generic implementation could be.


Embedding foreign languages
---------------------------

Code expressions may be written in any language, thus giving the
programmer instant access to the expressivity of that language.

For example, one may write ::

  <|sql| select id, name, address from users where age>=@MinimumAge |>

instead of concatenating strings or fiddling with a representation of
the above query using Erlang terms.

By extension, the same mechanism allows to embed any structured
textual data in its native format, not only code, for example XML::

  <xml|
    <person id="funnyFace">
      <name>
        <firstname>Bo</firstname>
        <lastname>Brady</lastname>
      </name>
    </person>
  |>

and use this notation even for matching.


Easier to write parse transforms
--------------------------------

If the "foreign" embedded language is Erlang itself, then writing code
that processes code becomes just as simple as writing regular code.
Compare ::

    [{match,
     1,
     {var,1,'Z'},
     {'fun',
         1,
         {clauses,
             [{clause,
                  1,
                  [{var,1,'X'}],
                  [],
                  [{tuple,
                       1,
                       [{var,1,'X'},
                        {call,
                            1,
                            {remote,1,{atom,1,erlang},{atom,1,now}},
                            []}]}]}]}}}]

with ::

    <| Z = fun(X)->{X, erlang:now()} end |>


True macros
-----------

Taking the next step with Erlang as an embedded language, one can
implement a macro system where the macros are not only simple textual
substitution mechanisms, but have the full power of the language at
their disposal.  This allows high level ideas to be expressed very
concisely and thus free the programmer from having to deal with
boilerplate code.


Specification
=============

Operators and Syntax
--------------------

The operators needed to implement metaprogramming facilities according
to [1]_ (where the description is adapted from) are described below.
Please note that in order to simplify the presentation, the values
shown for code expressions differ from actual values that will be
returned by an implementation.

Bracket
'''''''

Brackets can be inserted around any text to mark it as a code
fragment, represented at runtime as a parse tree.  A bracketed
expression can also be looked at as having a delayed execution time
(if the respective language is executable).

**Syntax**::

  <|Language:Func| Code |>
  <|Language| Code |>
  <|:Func| Code |>
  <| Code |>

``Language`` and ``Func`` are unquoted atoms.  There may be no
whitespace betwee the leftmost "``|``" characters.  ``Code`` may be
any text, with the only requirement that any "``<|``" and "``|>``"
inside it are balanced.  Otherwise, these tokens must be escaped like
"``\\<|``" or "``\\|>``" and the lexer will remove the escape
characters.  It is of course assumed that ``Code`` is a piece of valid
code in the language ``Language``.

In the shorter versions, the implied value for ``Language`` is ``erl``
and refers of course to Erlang itself.  The default value for ``Func``
is language-dependent, see below.

**Example**::

  > V = <| 5 + 3 |>.
  {op, '+', [{integer, 5}, {integer, 3}]}

Note that there is no separate type for code expressions, at run-time
they are indistinguishable from manually constructed parse trees.  It
would be useful to be able to distingush code expressions, not the
least so that the values could be printed to the console in a prettier
way::

  > V = <| 5 + 3 |>.
  <| 5 + 3 |>

That is however something left for future improvements.

Splice
''''''

Splice allows the combination of smaller code fragments to construct
larger ones.  This combination is achieved by "splicing-in" the
argument of the Splice in the context of the surrounding Brackets.

Splice combines snippets efficiently in the sense that the combination
of the subcomponents of the new computation is performed while the new
computation is being constructed, rather than while it is being
executed.

**Syntax**::

  ~Expression

The operator is interpreted slightly differently if it is encountered
inside a pattern or a regular expression.  Inside a regular
expression, it evaluates ``Expression`` and splices the result into
the body of a surrounding bracketed expression. Inside a pattern,
Expression must be a variable and the splice operator protects it from
the effect of bracketing.  More details later.

The value of ``Expression`` must be code based on the same language as
the surrounding brackets. Otherwise, the result is not defined.

The exact syntax for the splice operator is language dependent, there
isn't a single notation that would fit all possible languages.  In
case the default notation described here can't be used for a specific
language, the language support must provide an implementation of a
function named ``get_splice_syntax/0`` that returns a string
describing the syntax to look for. The argument of the splice operator
is specified as ``$$`` and any matching bracker-style characters
``{[((]}`` will be matched in the input too. A textual ``$$`` text has
to be escaped.

**Example**::

  > V = <| 5 + 3 |>.
  {op, '+', [{integer, 5}, {integer, 3}]}
  > <| {~V, ~V} |>.
  {tuple, [{op, '+', [{integer, 5}, {integer, 3}]},
           {op, '+', [{integer, 5}, {integer, 3}]}]}
  > foo_lang:get_splice_syntax().
  "bar{$$}"


Run
'''

Run allows the execution of a code fragment.  Having Run in the
language is important if we want to use code constructed using the
other Erlang constructs, without going outside the language.

Run is only meaningful for languages that represent code. The language
support must include appropriate mechanisms in the form of a ``run/1``
function that will get called with the respective argument.

**Syntax**::

  run(Expression)

**Example**::

  > V = <| 5 + 3 |>.
  {op, '+', [{integer, 5}, {integer, 3}]}
  > run(V).
  8

Lift
''''

Lift allows us to inject values of ground type into a value of type
code.  Ground types are types containing no arrows (function types);
that is because given a functional value we can't reconstruct its
source code representation.  However, functions can still be delayed
using Brackets::

  > F = fun(X) -> X+1 end.
  > V = <| F(5) |>.
  {call, F, [5]}
  > run(V).
  6

Lift is only meaningful for executable code. [[TODO: define language
support defining the syntax of lift]]

While both Brackets and Lift construct code, Lift does not delay its
argument.  Lift first evaluates its argument, then constructs a
representation for this value.

**Syntax**::

  lift(Expression)

**Example**::

  > V = lift(5 + 3).
  {integer, 8}

Variables and Levels
--------------------

If bracketed expressions are embedded inside other bracketed
expressions, the resulting code becomes *multi-level*.  Escaped
expressions have to be interpreted at the appropriate level and there
are two big issues to discuss.

Cross-Stage Persistence
'''''''''''''''''''''''

It feels natural to be able to write code like the following (where we
will use the "pretty" representation of code to increase readability)
::

  > A = 42, Code = <| fun(X) -> X + A end |>.
  <| fun(X) -> X + 42 |>

where A (defined at level 0) is treated as a free variable and thus
appears as a constant inside the code (at level 1).

Cross-Stage Safety
''''''''''''''''''

A variable violates cross-stage safety when it is bound at one level
and is used at a lower level. For example ::

  fun(X) -> <| fun(Y) -> ~(X+Y) |>

The annotations in this expression dictate computing X+Y in the first
stage, when the value of Y will be available only in the second stage!

In this implementation, the result will be a "variable not bound"
compilation error.

Applying Staging (an Example)
-----------------------------

Using these annotations, a programmer can modify the default strict
order evaluation of programs and modify a "normal" program to a staged
one. Let's take for example a function that searches a list for a value::

  %% member(term(), list()) -> boolean()
  member(_, []) -> false;
  member(V, [H|T]) ->
    if V==H -> true;
       true -> member(V, T)
    end.

We suppose the second argument, the list to be searched, is available
in the first stage and the other values (the first argument and the
return value) are delayed. This information can be used to apply
annotations so that the types of the values are kept consistent. The
result would be ::

  %% member(code(), list()) -> code()
  member2(_, []) -> <| false |>;
  member2(V, [H|T]) ->
    <| if ~V==~(lift(H)) -> true;
          true -> ~member2(V, T)
       end|>.

The result of calling ``<| fun(X) -> ~member2(X, [1, 2, 3]) end |>``
will be ::

  <| fun(X) ->
       if X==1 -> true;
          true -> if X==2 -> true;
                     true -> if X==3 -> true;
                                true -> false
                             end
                  end
       end
     end |>

Unfortunately, if using pattern matching in the function's clauses, I
couldn't find a way to perform staging without having to rewrite the
compiler. Work continues in this direction.


----


Parsing
-------

The construct described above will be parsed according to the
following rules:

- The code construct will be parsed as an expression *(TBD: define
  precedence)*.

- The handling of the code is delegated to a parser module, whose name
  is ``Language ++ "_parser"``.  If this module isn't found, an error
  is returned.

- The parser module must implement the following interface:

  * ``parse(Tokens) -> {ok, Tree} | {error, Error}``

    parses the given list of tokens and return a parse tree or an error.

  * ``get_default_func() -> string()`` [optional]

    returns the default value of Func.  If the function doesn't exist,
    an empty string is used.  This allows, for example, to specify if
    the Erlang parser will use ``parse_forms`` or ``parse_exprs`` as
    default.

  * ``parse_<Func>(Tokens) -> {ok, Tree} | {error, Error}`` [optional]

    If the code expression is declared with ``<|Lang:Func|``, then
    ``parse_Func/1`` will be invoked instead of ``parse/1``.

- The parse trees are composed of nodes represented as tuples.  The
  layout must be similar to the Erlang abstract format, with the node
  name as first element and a position indicator as the second.

- The MetaErlang parser takes the resulting parse tree and use it as
  the value of the code expression (after abstracting its
  representation).

- If the code expression is a pattern, it is necessary to replace the
  line numbers (or equivalent) with ``'_'``, so that different
  formatting styles of the same code do match.


Code Variables
--------------

When matching code expressions, one usually wants to extract
subexpressions.  This means that there must be a way to specify that
some entity in the code is just a placeholder.  The syntax for doing
this is language dependent, for Erlang the notation

    ``@VariableName``

is chosen. The notation ``@_`` will correspond to an unnamed variable.

If the language parser doesn't recognize code variables, then they
aren't supported for that language and their usage will yield an
exception.

When encountering a code variable, the language parser will return a
tree node ``{<code_variable_node_name>, Line, VariableName}``, where
``<code_variable_node_name>`` is an atom as returned by the language
support's ``get_code_variable_node/0`` API.  The MetaErlang parser
will detect these nodes and when abstracting the parse tree it will
replace them with a variable definition.

Taking the Erlang example above, writing ::

    <| @_ = fun(X) -> @Exprs end |> =
        <| Z = fun(X)->{X, erlang:now()} end |>``

will be equivalent to (rewritten as two matches for brevity) ::

    _ = {var, 1, 'Z'},
    Exprs = [{tuple,
              1,
              [{var,1,'X'},
               {call,
                1,
                {remote,1,{atom,1,erlang},{atom,1,now}},
                []}]}]

Of course, getting the results one wants requires deep knowledge of
the way the parse trees look like behind the simple-looking
syntax. Some things are even be impossible without further extensions,
so they will still require good ol' hands-on work (^^)/

Language support API summary
============================

The language support for language ``mylang`` is accessible from a
module named ``mylang_parser``. The API exposed by this module is as
follows:

  * ``get_splice_syntax() -> string()`` [optional]

    returns a string illustrating the syntax of the splice operator,
    with the argument represented as ``$$``.  Any ``{[()]}``
    characters must be balanced and will be matched accordingly in the
    input.  If not implemented, the default value of ``"~$$"`` will be
    assumed.

  * ``parse(Tokens) -> {ok, Tree} | {error, Error}``

    parses the given list of tokens and return a parse tree or an error.

  * ``get_default_func() -> string()`` [optional]

    returns the default value of Func.  If the function doesn't exist,
    an empty string is used.  This allows, for example, to specify if
    the Erlang parser will use ``parse_forms`` or ``parse_exprs`` as
    default.

  * ``parse_<Func>(Tokens) -> {ok, Tree} | {error, Error}`` [optional]

    If the code expression is declared with ``<|Lang:Func|``, then
    ``parse_Func/1`` will be invoked instead of ``parse/1``.

  * ``get_code_variable_node() -> atom()`` [optional]

    If the language supports variable bindings, the parser will return
    a "fake" node with this atom as name, which will be handled
    accordingly by the MetaErlang parser.  The default value is
    ``'@code_variable'``.

  * ``run(Expression) -> term()`` [optional]

    If the language is executable, this function evaluates the
    Expression (which must represent code in thet language) and
    returns the result as an Erlang term.

Open Issues
===========

- If the embedded language is Erlang, one might want to be able to use
  the code construct recursively.  This is not currently supported,
  but there is ongoing work in that direction.

- Some parsers may need extra parameters.  The above construct could
  be extended to allow for that, but the notation shouldn't be clumsy.

Reference Implementation
========================

A reference implementation exists, but it is incomplete. It will be
made available after processing the first feedback on this EEP, as I
suspect there will be substantial changes.


References
==========

.. [1] Multi-stage programming, Walid Taha and others
       http://www.cs.rice.edu/~taha/MSP/


Credits
=======

A lot of input comes from discussions with Richard O'Keefe, Ulf Wiger,
Yariv Sadan, Joe Armstrong and others.


Copyright
=========

This document has been placed in the public domain.


..
    Local Variables:
    mode: indented-text
    indent-tabs-mode: nil
    sentence-end-double-space: t
    fill-column: 70
    coding: utf-8
    End:


From kenneth.lundin@REDACTED  Tue Oct 23 13:57:39 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Tue, 23 Oct 2007 13:57:39 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
Message-ID: <de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>

Hi,

Just to clarify.
You happens to hate XML handling with the XMERL application that is
part of the Erlang/OTP distribution.
We are very well aware of that XMERL and the support for XML in the
distribution needs improvements and they will happen.

But I don't think you can draw any conclusions regarding Erlang as a
language and
how well suited it is for XML programming because of this.

I would appreciate if  you maybe can be a little more specific about
what you think needs to be improved in XMERL.
The output from xmerl_scan:file/2 is big but is not intended for
viewing on the screen. We think there is need for a more compact
output as well, among other things for performance reasons.

/Kenneth Erlang/OTP group at Ericsson

On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
> I only have so much horizontal real-estate on my screen and I totally
> hate how xmlElement and xmlAttribute take so much of it. I also hate
> how XML parsing code looks in Erlang.
>
> Am I the only one?
>
> I think Simon Peyton-Jones (Haskell) said that the syntax is the user
> interface of a language. If so then Erlang has a user interface only
> a mother could love, at least when it comes to parsing XML!
>
>        Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From lenartlad@REDACTED  Tue Oct 23 14:07:20 2007
From: lenartlad@REDACTED (Ladislav Lenart)
Date: Tue, 23 Oct 2007 14:07:20 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
Message-ID: <471DE3F8.3080903@volny.cz>

Joel Reymont wrote:
> I only have so much horizontal real-estate on my screen and I totally  
> hate how xmlElement and xmlAttribute take so much of it. I also hate  
> how XML parsing code looks in Erlang.
> 
> Am I the only one?

Nope, you are not. And from the record names it seems you are using xmerl.
I have to admit that I never got my head around it... However I have more
than positive experience with erlsom library so might take a look at it.

Hope this helps,

Ladislav Lenart




From dmitriid@REDACTED  Tue Oct 23 14:02:46 2007
From: dmitriid@REDACTED (Dmitrii 'Mamut' Dimandt)
Date: Tue, 23 Oct 2007 15:02:46 +0300
Subject: [erlang-questions] {eep} Metaprogramming in Erlang
In-Reply-To: <95be1d3b0710230447i3e6decf4se4fde0064aeb7005@mail.gmail.com>
References: <95be1d3b0710230447i3e6decf4se4fde0064aeb7005@mail.gmail.com>
Message-ID: <471DE2E6.8090607@gmail.com>

Vlad Dumitrescu wrote:
> Hello everybody,
>
> I have been working on some ideas for quite a while and it didn't go
> as fast as I would have liked, but now at least I have a draft that I
> am happy with.
>
> What I have in mind is (as the title says) adding metaprogramming
> capabilities to Erlang.
>
> The attached draft EEP is addressing some of the issues involved.
> Since this offers a solution to some problems ventilated more or less
> recently (last in row being Joel's XML rant :-), my belief is that it
> might be an interesting topic to discuss.
>
> The draft eep is inlined in this message (instead of being attached)
> so that it will be easier to make comments next to the relevant part
> of the document.
>
> I welcome all input, so please don't feel shy!
>   
Is it just me or this EEP tries to fast forward Erlang past Nemerle
(http://nemerle.org/Main_Page)?  :)


From vladdu55@REDACTED  Tue Oct 23 14:15:09 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 23 Oct 2007 14:15:09 +0200
Subject: [erlang-questions] {eep} Metaprogramming in Erlang
In-Reply-To: <471DE2E6.8090607@gmail.com>
References: <95be1d3b0710230447i3e6decf4se4fde0064aeb7005@mail.gmail.com>
	<471DE2E6.8090607@gmail.com>
Message-ID: <95be1d3b0710230515p5cc4c285j7744a7586d9f9d73@mail.gmail.com>

On 10/23/07, Dmitrii 'Mamut' Dimandt <dmitriid@REDACTED> wrote:
> Vlad Dumitrescu wrote:
> > What I have in mind is (as the title says) adding metaprogramming
> > capabilities to Erlang.

> Is it just me or this EEP tries to fast forward Erlang past Nemerle
> (http://nemerle.org/Main_Page)?  :)

Well, why not? :-) Especially as the working project name is Merlin,
which is by all means a stronger wizard than Nemerle ;-)

/Vlad


From ola.a.andersson@REDACTED  Tue Oct 23 14:25:19 2007
From: ola.a.andersson@REDACTED (Ola Andersson A)
Date: Tue, 23 Oct 2007 14:25:19 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
Message-ID: <148408C0A2D44A41AB295D74E1839975036E80C0@esealmw105.eemea.ericsson.se>

I agree! Except for the "Erlang" part...
xmerl is extremely unpleasant to work with, but isn't the problem more
related to XML than the programming language?
I find it hopeless to translate the XML mess into an accessible data
format.
I long for the good old ASN.1 days when protocols were designed for
computers
instead of humans. Who needs to parse protocols without a computer?
/OLA. 

>-----Original Message-----
>From: erlang-questions-bounces@REDACTED 
>[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Joel Reymont
>Sent: den 23 oktober 2007 12:52
>To: erlang-questions Questions
>Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
>
>I only have so much horizontal real-estate on my screen and I totally  
>hate how xmlElement and xmlAttribute take so much of it. I also hate  
>how XML parsing code looks in Erlang.
>
>Am I the only one?
>
>I think Simon Peyton-Jones (Haskell) said that the syntax is the user  
>interface of a language. If so then Erlang has a user interface only  
>a mother could love, at least when it comes to parsing XML!
>
>	Joel
>
>--
>http://wagerlabs.com
>
>
>
>
>
>_______________________________________________
>erlang-questions mailing list
>erlang-questions@REDACTED
>http://www.erlang.org/mailman/listinfo/erlang-questions
>


From joelr1@REDACTED  Tue Oct 23 14:36:24 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 13:36:24 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>
Message-ID: <00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>

Kenneth,

On Oct 23, 2007, at 12:57 PM, Kenneth Lundin wrote:

> But I don't think you can draw any conclusions regarding Erlang as a
> language and how well suited it is for XML programming because of  
> this.

Take a look at the following [1] and try to visualize an  
implementation in Erlang. More thoughts after the example.

The data:

<Export>
   <Product>
     <SKU>403276</SKU>
     <ItemName>Trivet</ItemName>
     <CollectionNo>0</CollectionNo>
     <Pages>0</Pages>
   </Product>
</Export>

The Ruby hPricot code:

FIELDS = %w[SKU ItemName CollectionNo Pages]

doc = Hpricot.parse(File.read("my.xml"))
(doc/:product).each do |xml_product|
   product = Product.new
   for field in FIELDS
     product[field] = (xml_product/field.intern).first.innerHTML
   end
   product.save
end

This dovetails with the metaprogramming EPP that Vlad has just  
submitted. Erlang is mind-numbingly rigid in its syntax. I cannot  
emphasize this more!!!

The simplest example is that the "synchronous message passing"  
notation of !! (double exclamation point). It could be used to neatly  
translate the above into XmlProduct !! {search, Field !! intern} !!  
first !! innerHTML. Alas, this is impossible even with a parse  
transform since there's no !! in Erlang.

Any suggestions to using call(...) instead of !! will only prove my  
point about the ugly user interface.

Why does it upset me so much you may ask? It's very simple! Erlang as  
a huge unstoppable HTML and XML transformation engine. Throw together  
a bunch of boxes and you can start sucking in web pages and spitting  
out RSS feeds, rivers of news, etc.

Erlang enables mashups on a large scale but this type of application  
needs a friendly user interface, one that seems to be impossible to  
build right now due to syntax limitations!

	Thanks, Joel

[1] http://errtheblog.com/post/8

--
http://wagerlabs.com



From joelr1@REDACTED  Tue Oct 23 14:42:44 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 13:42:44 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <471DE3F8.3080903@volny.cz>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
Message-ID: <7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>


On Oct 23, 2007, at 1:07 PM, Ladislav Lenart wrote:

> And from the record names it seems you are using xmerl.

It's just an easy target to pick on.

> However I have more
> than positive experience with erlsom library so might take a look  
> at it.

I did this morning and discarded it immediately as it wants me to  
supply schema for everything I want to parse. I'm interested in HTML  
scraping and mashups and erlsom does not seem to be useful for this  
purpose.

	Thanks, Joel

--
http://wagerlabs.com







From kevin@REDACTED  Tue Oct 23 14:56:08 2007
From: kevin@REDACTED (Kevin A. Smith)
Date: Tue, 23 Oct 2007 08:56:08 -0400
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
Message-ID: <D6288411-04C6-4F44-BEAC-532F0520D413@hypotheticalabs.com>

If you can get by with a SAX-based approach erlsom can work without  
schema.

Atomizer (http://code.google.com/p/atomizer/) uses this approach to  
parse Atom feeds.

--Kevin
On Oct 23, 2007, at 8:42 AM, Joel Reymont wrote:

>
> On Oct 23, 2007, at 1:07 PM, Ladislav Lenart wrote:
>
>> And from the record names it seems you are using xmerl.
>
> It's just an easy target to pick on.
>
>> However I have more
>> than positive experience with erlsom library so might take a look
>> at it.
>
> I did this morning and discarded it immediately as it wants me to
> supply schema for everything I want to parse. I'm interested in HTML
> scraping and mashups and erlsom does not seem to be useful for this
> purpose.
>
> 	Thanks, Joel
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From vladdu55@REDACTED  Tue Oct 23 15:02:45 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Tue, 23 Oct 2007 15:02:45 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
Message-ID: <95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>

Hi Joel,

On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
> I'm interested in HTML
> scraping and mashups and erlsom does not seem to be useful for this
> purpose.

Do you try to scrape arbitrary HTML? I don't think a XML parser will
help that much in such a case, because HTML is only a distant cousin
of XML...

regards,
Vlad


From joelr1@REDACTED  Tue Oct 23 15:08:13 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 14:08:13 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
Message-ID: <9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>



On Oct 23, 2007, at 2:02 PM, Vlad Dumitrescu wrote:
> Do you try to scrape arbitrary HTML? I don't think a XML parser will
> help that much in such a case, because HTML is only a distant cousin
> of XML...

Completely arbitrary HTML. Any web page out there. The syntax and  
approach won't be much for HTML, assuming you had a robust parser. My  
rant is about the syntax.

--
http://wagerlabs.com







From sean.hinde@REDACTED  Tue Oct 23 15:30:24 2007
From: sean.hinde@REDACTED (Sean Hinde)
Date: Tue, 23 Oct 2007 14:30:24 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
Message-ID: <54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>

Take a look at yaws_html.erl. That is quite a nice parser that  
doesn't produce the same bloat as xmerl

Sean

On 23 Oct 2007, at 14:08, Joel Reymont wrote:

>
>
> On Oct 23, 2007, at 2:02 PM, Vlad Dumitrescu wrote:
>> Do you try to scrape arbitrary HTML? I don't think a XML parser will
>> help that much in such a case, because HTML is only a distant cousin
>> of XML...
>
> Completely arbitrary HTML. Any web page out there. The syntax and
> approach won't be much for HTML, assuming you had a robust parser. My
> rant is about the syntax.
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From joelr1@REDACTED  Tue Oct 23 15:46:34 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 14:46:34 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
Message-ID: <04618A72-BCE8-46EA-9E05-71D749BB97B1@gmail.com>


On Oct 23, 2007, at 2:30 PM, Sean Hinde wrote:

> Take a look at yaws_html.erl. That is quite a nice parser that  
> doesn't produce the same bloat as xmerl

Are there any examples of using yaws_html as well as the output that  
it produces? Would be nice to include in this thread.

	Thanks, Joel

--
http://wagerlabs.com







From kevin@REDACTED  Tue Oct 23 15:46:49 2007
From: kevin@REDACTED (Kevin A. Smith)
Date: Tue, 23 Oct 2007 09:46:49 -0400
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
Message-ID: <09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>

Maybe that should be packaged separately? Seems odd that you'd have  
to get the webserver just for a HTML parser....

FWIW, I tried writing a very permissive feedparser but lost interest  
partially due to the ugliness of Erlang's XML parsing APIs.

--Kevin

--Kevin
On Oct 23, 2007, at 9:30 AM, Sean Hinde wrote:

> Take a look at yaws_html.erl. That is quite a nice parser that
> doesn't produce the same bloat as xmerl
>
> Sean
>
> On 23 Oct 2007, at 14:08, Joel Reymont wrote:
>
>>
>>
>> On Oct 23, 2007, at 2:02 PM, Vlad Dumitrescu wrote:
>>> Do you try to scrape arbitrary HTML? I don't think a XML parser will
>>> help that much in such a case, because HTML is only a distant cousin
>>> of XML...
>>
>> Completely arbitrary HTML. Any web page out there. The syntax and
>> approach won't be much for HTML, assuming you had a robust parser. My
>> rant is about the syntax.
>>
>> --
>> http://wagerlabs.com
>>
>>
>>
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From joelr1@REDACTED  Tue Oct 23 16:01:11 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 15:01:11 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
Message-ID: <93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>


On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:

> FWIW, I tried writing a very permissive feedparser but lost  
> interest partially due to the ugliness of Erlang's XML parsing APIs.

Running yaws_html:parse/1 on a sample RSS feed works just fine. I  
suspect you can't get anymore permissive than that.

--
http://wagerlabs.com







From ulf.wiger@REDACTED  Tue Oct 23 16:09:58 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Tue, 23 Oct 2007 16:09:58 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
Message-ID: <471E00B6.2030303@ericsson.com>

Joel Reymont wrote:
> I only have so much horizontal real-estate on my screen and I totally  
> hate how xmlElement and xmlAttribute take so much of it. I also hate  
> how XML parsing code looks in Erlang.

The original idea with xmerl (dating back to 2001), was
to map XML to S-expressions. The records came about when
trying to support namespaces, XPATH, et al.

There may be some gotchas in what assumptions xmerl_scan
makes about the hook functions, but at least for simple
XML, you can call xmerl_scan with your own hook function:

17>xmerl_scan:file(".../xmerl-0.18.1/priv/testdata/test3.xml",
     [{hook_fun,
       fun(#xmlElement{name = N,
                       attributes = As,
                       content = C},S) ->
              As1 = [{K,V} ||
                     #xmlAttribute{name=K,
                                   value=V} <- As],
              {{N,As1,C},S};
          (#xmlText{value = V}, S) ->
              {V,S}; (X,S) -> {X, S} end}]). 
     {{'People',[],
            ["\n  ",
             {comment,[],["This is a comment"]},
             "\n  ",
             {'Person',[{'Type',"Personal"}],["\n  "]},
             "\n"]},
  []}

BR,
Ulf W


From erlang@REDACTED  Tue Oct 23 16:13:27 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Tue, 23 Oct 2007 16:13:27 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
Message-ID: <9b08084c0710230713i1d104ea1y1ba2866f89687762@mail.gmail.com>

I have written several XML parsers in several states of completeness,
at the moment I'm trying to put together yet another xml toolkit (I
might actually release
this one)

The problem with "parsing" xml is not so much the parsing but what you want to
do with the parse tree. Do you need validation? how are the DTD
defined and so on.

Here are are some of the questions that occur to me in the design of an
XML parser.

1) Is the input small or large. small means will fit into memory.

    In the case of small input then everything fits into memory - I
can happily parse
    input streams of a 200K lines.

    The only large files XML I've found are a tens of Gigabytes of
data - is it important
    to be able to parse and validate these? or do you just want sax
like processing

2) Is the input stream "framed" - ie we have a framing protocol so we
know we have
    and entire XML document or do you want a re-entrant parser.

3) Do you need to handle streams of xml terms. This is often problematic
    since nobody can agree on the framing protocol - does each term begin
    with a new <?xml ...?> header?

4) Do you want validation. In which case how do you find the DTD/schema - do you
    have to comply with OASIS catalogues?
    Do you want DTD's, Schema, RNC, (or an erlang ad hock grammar)

    (My solution is to parameterize the parser with a fun F - F(URI)
is a function
     that knows how to find the DTD in URI - the OASIS catalogue structure
     is not something that I want to be concerned with - most
applications seem to
     titall ignore this)

5) Do you want the parser to try an correct errors and recover, or
bail out early.

6) Do you want unicode support? or just ASCII.

7) Does the data come from files, sockets, binaries?

8) Do you want strict or lazy parsing. If you don't look an attribute
you might like to
    defer parsing the content until you actually need it.

9) Do you want to check all ids and idrefs and correctly handle
NOTATION's  and so on
    i.e. all the weird things in the XML spec that 99.9% of
programmers have never used.

10) In the case on XML without a DTD do you want a heuristic to throw away
      non-significant white space (I often use a simple heuristic if all the
      PCDATA children of a tag are all white space then throw all this
white space away)

...

It's very difficult to write a parser that correctly handles *all* of
these and is fast, small etc.

I have made a set of compromises and a toolkit that provides the following.

   1) A tag level interface to the system
       this has a file type interface.

        Pid = open_xml_token_stream(Descriptor)

      makes a re-entrant token scanner

       get_next_token(Pid) -> Token | eof

      A lot of things can be done with this alone, for example a SAX
like processor

    2) A simple parser this takes a token stream and parses it just checking for
        well formed-ness.

    3) A validator that runs on the output of 2 - this only
understands DTDs (not schemas,
       or rnc)

    There are also diverse routines to parse files etc, based on these.

    I've also written an XSLT type thing that takes the output of 2)
or 3) and transforms
it.

     This is just ASCII.

     I've talked to a lot of people about XML - most people (the majority) want
something to parse a small ASCII file containing a single XML data structure.

      The data structure is well formed - there is no DTD - and they don't care
about integrity constraints on the attributes- They don't care about
entity expansion,
NOTATIONs CDATA etc.

     The kind of Ruby code shown in an earlier posting is easy given a
simple parse tree

     My experimental parser turns an XML data structure in a

      @type xml() = {node,Line,Tag,attrs(),[xml()]} |
{raw,Ln,AllBlack:bool(), bin()}

     It's easy to write a fold function that applies a Fun to each node

fold_over_nodes(Fun, Env, {node, _, _, _, C} = Node) ->
    Env1 = Fun(Node, Env),
    fold_over_nodes(Fun, Env1, C);
fold_over_nodes(Fun, Env, {raw,_,_,_} = Raw) ->
    Fun(Raw, Env);
fold_over_nodes(Fun, Env, [H|T]) ->
    Env1 = fold_over_nodes(Fun, Env, H),
    fold_over_nodes(Fun, Env1, T);
fold_over_nodes(Fun, Env, []) ->
    Env.

   (like foldl) - this can be used to extract tags

     F = fun({node,_,Stag,_,C}=N, E) ->
            case member(Stag, [a,b,c]) of
                true -> [N|E];
                false -> E
            end,
     Tags = fold_over_nodes(Tree, [], F)

For very simple applications I could put together a parser that does
the following:

   1) Small files only.
   2) no DTDs or grammar checking
   3) white space normalisation according to the following.
       If a tag has MIXED content (ie tags and PCDATA) and all the
PCDATA bits are all
       blank then remove all the PCDATA
   4) ASCII
   5) simple "obvious" parse tree {tag,Line,Name,[Attr], [Children]}
       Attr is a sorted [{Key,Val}] list
   6) Simple SAX library, foldtree functions, find object in tree (like Xpath
       only in Erlang)

I have a 6-pack of parsers than almost do this - they are all specialsed in
different ways for infinite streams and so on ...

I'm not sure if a general purpose toolkit (that allows you to build
the above) or a set
of completely different parsers with different properties is desirable.

/Joe Armstrong



On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
>
>
> On Oct 23, 2007, at 2:02 PM, Vlad Dumitrescu wrote:
> > Do you try to scrape arbitrary HTML? I don't think a XML parser will
> > help that much in such a case, because HTML is only a distant cousin
> > of XML...
>
> Completely arbitrary HTML. Any web page out there. The syntax and
> approach won't be much for HTML, assuming you had a robust parser. My
> rant is about the syntax.
>
> --
> http://wagerlabs.com
>
>
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From anders.nygren@REDACTED  Tue Oct 23 16:27:49 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 23 Oct 2007 09:27:49 -0500
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
Message-ID: <d3520a2f0710230727v69a7f31o2a5925774d38ca74@mail.gmail.com>

On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
>
> On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
>
> > FWIW, I tried writing a very permissive feedparser but lost
> > interest partially due to the ugliness of Erlang's XML parsing APIs.
>
> Running yaws_html:parse/1 on a sample RSS feed works just fine. I
> suspect you can't get anymore permissive than that.

I tried to use it a couple of years ago and it was of no help to me since
it actually requires correct HTML. Which the sites I tried to scrape
refused to provide, (missing end tags and so on).

Anders


From hakan@REDACTED  Tue Oct 23 16:47:46 2007
From: hakan@REDACTED (Hakan Mattsson)
Date: Tue, 23 Oct 2007 16:47:46 +0200 (CEST)
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>
	<00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
Message-ID: <Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>

On Tue, 23 Oct 2007, Joel Reymont wrote:

JR> Take a look at the following [1] and try to visualize an  
JR> implementation in Erlang. More thoughts after the example.
JR> 
JR> The data:
JR> 
JR> <Export>
JR>    <Product>
JR>      <SKU>403276</SKU>
JR>      <ItemName>Trivet</ItemName>
JR>      <CollectionNo>0</CollectionNo>
JR>      <Pages>0</Pages>
JR>    </Product>
JR> </Export>
JR> 
JR> The Ruby hPricot code:
JR> 
JR> FIELDS = %w[SKU ItemName CollectionNo Pages]
JR> 
JR> doc = Hpricot.parse(File.read("my.xml"))
JR> (doc/:product).each do |xml_product|
JR>    product = Product.new
JR>    for field in FIELDS
JR>      product[field] = (xml_product/field.intern).first.innerHTML
JR>    end
JR>    product.save
JR> end

At a first glance your Ruby code looks impressively
compact.  But the corresponding implementation in
Erlang is about the same size. What's the point in
adding some syntactic sugar in order to make it even
more compact? It is just a matter of taste.

    % cat product.erl
    -module(product).
    -compile(export_all).
    -include_lib("xmerl/include/xmerl.hrl").
    
     parse(File) ->
        {#xmlElement{content = Exports}, _} = xmerl_scan:file(File),
        [{Tag, Val} || #xmlElement{content = Products} <- Exports,
                       #xmlElement{content = Fields} <- Products,
                       #xmlText{parents = [{Tag, _} | _], value = Val}  <- Fields].
    
    % erl
    Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false]
    
    Eshell V5.5.5  (abort with ^G)
    1> product:parse("my.xml").
    [{'SKU',"403276"},{'ItemName',"Trivet"},{'CollectionNo',"0"},{'Pages',"0"}]
    2> 

/H?kan

From kevin@REDACTED  Tue Oct 23 16:51:02 2007
From: kevin@REDACTED (Kevin A. Smith)
Date: Tue, 23 Oct 2007 10:51:02 -0400
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
Message-ID: <A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>

Possibly. My understanding was that it still required well-formed  
documents to function. A lot of feeds feature varying amounts of  
"well-formedness", sadly.

--Kevin
On Oct 23, 2007, at 10:01 AM, Joel Reymont wrote:

>
> On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
>
>> FWIW, I tried writing a very permissive feedparser but lost  
>> interest partially due to the ugliness of Erlang's XML parsing APIs.
>
> Running yaws_html:parse/1 on a sample RSS feed works just fine. I  
> suspect you can't get anymore permissive than that.
>
> --
> http://wagerlabs.com
>
>
>
>
>



From erlang@REDACTED  Tue Oct 23 16:58:43 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Tue, 23 Oct 2007 16:58:43 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <d3520a2f0710230727v69a7f31o2a5925774d38ca74@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<d3520a2f0710230727v69a7f31o2a5925774d38ca74@mail.gmail.com>
Message-ID: <9b08084c0710230758n75016e23mdfabad2671b0bb0c@mail.gmail.com>

This indicates that you don't want an XML parser. If the XML (HTNL) is not
well formed then you probably just want a tag parser

My guess is that if you tokenise the input into a sequence of tags and
then pattern
match over the tags you'll get what you want.

The tokenised file looks like this

     [...,
      {sTag,a,[{href,"..."}]}
      {eTag, img,[{src,"..."}]},
      {eTag,img},
      {eTag,a},
      {stag,p,[]},
      {raw,"...."}
      ...
    ]

  Then you write pattens to extract the content
      ...

   this is described here

http://www.trapexit.org/forum/viewtopic.php?p=20670&highlight=&sid=ab39db1f70f1a3a68602f830091ea547

   From what has been posted I get the following picture

   1) There are lots of XML libraries around (I have a 6-pack)
       other people have mentioned libraries that I was unaware of
   2) The code for these cannot be found in one place
   3) The documentation for how to use these is non-existent

    The solution is

    - move all code to one site
    - organise it
    - document it

    This is a lot of work -

/Joe



On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
> >
> > On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
> >
> > > FWIW, I tried writing a very permissive feedparser but lost
> > > interest partially due to the ugliness of Erlang's XML parsing APIs.
> >
> > Running yaws_html:parse/1 on a sample RSS feed works just fine. I
> > suspect you can't get anymore permissive than that.
>
> I tried to use it a couple of years ago and it was of no help to me since
> it actually requires correct HTML. Which the sites I tried to scrape
> refused to provide, (missing end tags and so on).
>
> Anders
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ulf.wiger@REDACTED  Tue Oct 23 17:05:32 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Tue, 23 Oct 2007 17:05:32 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <D6288411-04C6-4F44-BEAC-532F0520D413@hypotheticalabs.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>	<471DE3F8.3080903@volny.cz>	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<D6288411-04C6-4F44-BEAC-532F0520D413@hypotheticalabs.com>
Message-ID: <471E0DBC.8060205@ericsson.com>

Kevin A. Smith wrote:
> If you can get by with a SAX-based approach erlsom can work without  
> schema.
> 
> Atomizer (http://code.google.com/p/atomizer/) uses this approach to  
> parse Atom feeds.

Just to be clear, you could "easily" write a wrapper around
xmerl_scan that fits more or less exactly into the atom_parser
structure. This was pretty much the idea with xmerl, but the
documentation is terse enough that most people have missed
that.

Also, while there are some wrappers provided with xmerl,
there is no SAX wrapper.

Just to illustrate with an (incomplete) wrapper:

-module(xmerl_sax).

-export([file/2]).

-include_lib("xmerl/include/xmerl.hrl").

file(F, CB) when is_function(CB, 3) ->
     xmerl_scan:file(F, [{event_fun, fun(E, S) ->
					    event(E, CB, S)
				    end},
			{acc_fun, fun(_, Acc, S) ->
					  {Acc, S}
				  end}]).


event(#xmerl_event{event = E,
		   data = D}, CB, S) ->
     case D of
	#xmlPI{} -> S;
	#xmlComment{} -> S;
	#xmlDecl{} -> S;
	_ ->
	    ES = xmerl_scan:event_state(S),
	    ES1 = CB(E, data(D), ES),
	    xmerl_scan:event_state(ES1, S)
     end.

data(#xmlAttribute{name = N, value = V}) ->
     {attribute, N, V};
data(#xmlElement{name = N, attributes = As, content = C}) ->
     {element, N, [{K,V} || #xmlAttribute{name = K, value = V} <- As], C};
data(document) -> document;
data(#xmlText{value = V}) ->
     {text, V}.


11> 
xmerl_sax:file("/home/etxuwig/contribs/xmerl-0.18.1/priv/testdata/test3.xml",fun(E,Info,S) 
-> io:format("E = ~p, Info = ~p~n", [E,Info]), S end).
E = started, Info = document
E = ended, Info = {attribute,encoding,"iso-8859-1"}
E = started, Info = {element,'People',[],[]}
E = started, Info = {text,undefined}
E = ended, Info = {text,"\n  "}
E = started, Info = {element,comment,[],[]}
E = started, Info = {text,undefined}
E = ended, Info = {text,"This is a comment"}
E = ended, Info = {element,comment,[],[]}
E = started, Info = {text,undefined}
E = ended, Info = {text,"\n  "}
E = ended, Info = {attribute,'Type',"Personal"}
E = started, Info = {element,'Person',[{'Type',"Personal"}],[]}
E = started, Info = {text,undefined}
E = ended, Info = {text,"\n  "}
E = ended, Info = {element,'Person',[{'Type',"Personal"}],[]}
E = started, Info = {text,undefined}
E = ended, Info = {text,"\n"}
E = ended, Info = {element,'People',[],[]}
E = ended, Info = document



...then, xmerl spits out an xmlElement record anyway,
which is a bug, IMO.

Another bug is that you can't tell xmerl which accumulator
to use as the return value. This would be easily fixed.

I agree with Joe: it's pretty easy to write a limited
XML parser that handles > 90% of all XML out there and
returns something that's visually appealing.

Writing a few front-ends to xmerl that are leagues more
user friendly than the generic back-end is not rocket-
science, but I have no problem accepting that people
don't think they should have to do that.

BR,
Ulf W


From erlang@REDACTED  Tue Oct 23 17:09:11 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Tue, 23 Oct 2007 17:09:11 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
Message-ID: <9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>

I've seen some work on parsing badly formed HTML.

If I remember rightly you keep a stack of the currently open tags
then stacks for things like <font> <b> <i> tags etc. So you end up with
several small stacks. Each new open or close tag pushes or pops
things onto these stacks.

When you hit raw data you pattern match over the stacks to figure out
what to do.

As an aside it occurred to me that mozilla is probably pretty good at
sceen scraping (or whatever it's called) - so it should be possible to write
a Firefox extension to do this that talks through a socket to Erlang.
<somebody told me this was easy, but they obviously knew more than I do>

You could then use Erlang as a coordination language controlling
a load of firefoxes on different machines, telling them to go get pages and
scrape the pages for data which they send back to Erlang.

if we could use firefox as a component then we could avoid reinventing
the wheel (again)

/Joe


On 10/23/07, Kevin A. Smith <kevin@REDACTED> wrote:
> Possibly. My understanding was that it still required well-formed
> documents to function. A lot of feeds feature varying amounts of
> "well-formedness", sadly.
>
> --Kevin
> On Oct 23, 2007, at 10:01 AM, Joel Reymont wrote:
>
> >
> > On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
> >
> >> FWIW, I tried writing a very permissive feedparser but lost
> >> interest partially due to the ugliness of Erlang's XML parsing APIs.
> >
> > Running yaws_html:parse/1 on a sample RSS feed works just fine. I
> > suspect you can't get anymore permissive than that.
> >
> > --
> > http://wagerlabs.com
> >
> >
> >
> >
> >
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From joelr1@REDACTED  Tue Oct 23 17:23:30 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Tue, 23 Oct 2007 16:23:30 +0100
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
Message-ID: <3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>


On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:

> You could then use Erlang as a coordination language controlling
> a load of firefoxes on different machines, telling them to go get  
> pages and
> scrape the pages for data which they send back to Erlang.

This is nuts!!! /With all due respect to Joe/

--
http://wagerlabs.com







From thomasl_erlang@REDACTED  Tue Oct 23 16:27:19 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Tue, 23 Oct 2007 07:27:19 -0700 (PDT)
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <04618A72-BCE8-46EA-9E05-71D749BB97B1@gmail.com>
Message-ID: <606623.91435.qm@web38810.mail.mud.yahoo.com>


--- Joel Reymont <joelr1@REDACTED> wrote:

> 
> On Oct 23, 2007, at 2:30 PM, Sean Hinde wrote:
> 
> > Take a look at yaws_html.erl. That is quite a nice
> parser that  
> > doesn't produce the same bloat as xmerl
> 
> Are there any examples of using yaws_html as well as
> the output that  
> it produces? Would be nice to include in this
> thread.

1> yaws_html:parse("ShowLetter.html").
{ehtml,[],
       [{html,[],
              [{head,[],
                     [{title,[],"Yahoo! Mail -
thomasl_erlang@REDACTED"},
                      {script,[],
                              "\n<!-- \n\tif(typeof
top.frames[\"wmailmain\"] != \"undefined\")
window.open(\"http://mail.yahoo.com\", \"_top\");\n//
-->\n"},
                      {link,[{rel,"stylesheet"},
                            
{href,"ShowLetter_files/mail_blue_all.css"},
                             {type,"text/css"},
                             {media,"all"}]},
                     
{script,[{src,"ShowLetter_files/mailcommonlib.js"}],[]},
...
and so on

However, note that yaws_html (1.68 in this case)
apparently isn't robust enough to handle unclosed tags
and perhaps other nastiness. You get parse errors
instead, which might not be what you want for a real
html processor. Good luck.

(For extra credit, write an xmerl -> ehtml converter.)

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From bob@REDACTED  Tue Oct 23 17:30:25 2007
From: bob@REDACTED (Bob Ippolito)
Date: Tue, 23 Oct 2007 08:30:25 -0700
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
Message-ID: <6a36e7290710230830r32090986nddef00883c7f1f4d@mail.gmail.com>

http://tidy.sourceforge.net/ is the typical library I've seen used to
transform arbitrary HTML into a valid document quickly and without
re-inventing the wheel. Much easier than trying to integrate with
Mozilla.

-bob

On 10/23/07, Joe Armstrong <erlang@REDACTED> wrote:
> I've seen some work on parsing badly formed HTML.
>
> If I remember rightly you keep a stack of the currently open tags
> then stacks for things like <font> <b> <i> tags etc. So you end up with
> several small stacks. Each new open or close tag pushes or pops
> things onto these stacks.
>
> When you hit raw data you pattern match over the stacks to figure out
> what to do.
>
> As an aside it occurred to me that mozilla is probably pretty good at
> sceen scraping (or whatever it's called) - so it should be possible to write
> a Firefox extension to do this that talks through a socket to Erlang.
> <somebody told me this was easy, but they obviously knew more than I do>
>
> You could then use Erlang as a coordination language controlling
> a load of firefoxes on different machines, telling them to go get pages and
> scrape the pages for data which they send back to Erlang.
>
> if we could use firefox as a component then we could avoid reinventing
> the wheel (again)
>
> /Joe
>
>
> On 10/23/07, Kevin A. Smith <kevin@REDACTED> wrote:
> > Possibly. My understanding was that it still required well-formed
> > documents to function. A lot of feeds feature varying amounts of
> > "well-formedness", sadly.
> >
> > --Kevin
> > On Oct 23, 2007, at 10:01 AM, Joel Reymont wrote:
> >
> > >
> > > On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
> > >
> > >> FWIW, I tried writing a very permissive feedparser but lost
> > >> interest partially due to the ugliness of Erlang's XML parsing APIs.
> > >
> > > Running yaws_html:parse/1 on a sample RSS feed works just fine. I
> > > suspect you can't get anymore permissive than that.
> > >
> > > --
> > > http://wagerlabs.com
> > >
> > >
> > >
> > >
> > >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From tobbe@REDACTED  Tue Oct 23 17:57:05 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Tue, 23 Oct 2007 17:57:05 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>	<00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
	<Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
Message-ID: <ffl5ki$tmp$1@ger.gmane.org>

Hakan Mattsson wrote:
> On Tue, 23 Oct 2007, Joel Reymont wrote:
> 
> JR> Take a look at the following [1] and try to visualize an  
> JR> implementation in Erlang. More thoughts after the example.
> JR> 
> JR> The data:
> JR> 
> JR> <Export>
> JR>    <Product>
> JR>      <SKU>403276</SKU>
> JR>      <ItemName>Trivet</ItemName>
> JR>      <CollectionNo>0</CollectionNo>
> JR>      <Pages>0</Pages>
> JR>    </Product>
> JR> </Export>
> JR> 
> JR> The Ruby hPricot code:
> JR> 
> JR> FIELDS = %w[SKU ItemName CollectionNo Pages]
> JR> 
> JR> doc = Hpricot.parse(File.read("my.xml"))
> JR> (doc/:product).each do |xml_product|
> JR>    product = Product.new
> JR>    for field in FIELDS
> JR>      product[field] = (xml_product/field.intern).first.innerHTML
> JR>    end
> JR>    product.save
> JR> end
> 
> At a first glance your Ruby code looks impressively
> compact.  But the corresponding implementation in
> Erlang is about the same size. What's the point in
> adding some syntactic sugar in order to make it even
> more compact? It is just a matter of taste.
> 
>     % cat product.erl
>     -module(product).
>     -compile(export_all).
>     -include_lib("xmerl/include/xmerl.hrl").
>     
>      parse(File) ->
>         {#xmlElement{content = Exports}, _} = xmerl_scan:file(File),
>         [{Tag, Val} || #xmlElement{content = Products} <- Exports,
>                        #xmlElement{content = Fields} <- Products,
>                        #xmlText{parents = [{Tag, _} | _], value = Val}  <- Fields].
>     
>     % erl
>     Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false]
>     
>     Eshell V5.5.5  (abort with ^G)
>     1> product:parse("my.xml").
>     [{'SKU',"403276"},{'ItemName',"Trivet"},{'CollectionNo',"0"},{'Pages',"0"}]
>     2> 
> 
> /H?kan
> 

Well done H?kan ;-)

Here is another (not as nice as your) solution, which however is
rather fun, making use of Xpath:
--------------------------------------------
-module(xp).
-export([go/0, go/1]).

-include_lib("xmerl/include/xmerl.hrl").

-define(Val(X),
        (fun() ->
                 [#xmlElement{name = N, content = [#xmlText{value =
V}|_]}] = X,
                 {N,V} end)()).

go() ->
    go("/home/tobbe/hej.xml").

go(File) ->
    {Xml, _} = xmerl_scan:file(File),
    [?Val(xmerl_xpath:string("//SKU", Xml)),
     ?Val(xmerl_xpath:string("//ItemName", Xml)),
     ?Val(xmerl_xpath:string("//CollectionNo", Xml)),
     ?Val(xmerl_xpath:string("//Pages", Xml))].
-------------------------------------------------

5> xp:go().
[{'SKU',"403276"},{'ItemName',"Trivet"},{'CollectionNo',"0"},{'Pages',"0"}]


Cheers, Tobbe






From Peter.Chapin@REDACTED  Tue Oct 23 17:47:26 2007
From: Peter.Chapin@REDACTED (Peter C. Chapin)
Date: Tue, 23 Oct 2007 11:47:26 -0400
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <d3520a2f0710230727v69a7f31o2a5925774d38ca74@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>	<471DE3F8.3080903@volny.cz>	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<d3520a2f0710230727v69a7f31o2a5925774d38ca74@mail.gmail.com>
Message-ID: <471E178E.60604@vtc.vsc.edu>

Anders Nygren wrote:

> I tried to use it a couple of years ago and it was of no help to me since
> it actually requires correct HTML. Which the sites I tried to scrape
> refused to provide, (missing end tags and so on).
>   

It is not necessarily incorrect for an HTML document to have missing end
tags. For some elements the end tag is optional. Trying to parse an HTML
document with an XML parser is not likely to work well, however. One
must either use an SGML parser or make sure you only point your XML
parser at an XHTML document.

Peter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Peter.Chapin.vcf
Type: text/x-vcard
Size: 308 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/51a412d3/attachment.vcf>

From anders.nygren@REDACTED  Tue Oct 23 21:03:14 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 23 Oct 2007 14:03:14 -0500
Subject: [erlang-questions] widefinder update
Message-ID: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>

To summarize my progress on the widefinder problem
A few days ago I started with Steve Vinoski's tbray16.erl
As a baseline on my 1.66 GHz dual core Centrino
laptop, Linux,
tbray16
real    0m7.067s
user    0m12.377s
sys     0m0.584s

I removed the dict used for the shift table,
and changed the min_heap_size.
That gave
real    0m2.713s
user    0m4.168s
sys     0m0.412s

(see tbray_tuple.erl and wfbm4_tuple.erl)
Steve reported that it ran in ~1.9 s on his 8 core server.

Then I removed the dicts that were used for collecting the
matches and used ets instead, and got some improvement
on my dual core laptop.
real    0m2.220s
user    0m3.252s
sys     0m0.344s

(see tbray_ets.erl and wfbm4_ets.erl)

Interestingly Steve reported that it actually performed
worse on his 8 core server.

These versions all read the whole file into memory at the start.
On my laptop that takes ~400ms (when the file is already cached
in the OS).

So I changed it to read the file in chucks and spawn the worker
after each chunk is read.

tbray_blockread with 4 processes
real    0m1.992s
user    0m3.176s
sys     0m0.420s

(see tbray_blockread.erl and wfbm4_ets.erl)

Running it in the erlang shell it takes ~1.8s.

Just starting and stopping the VM takes
time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop

real    0m1.229s
user    0m0.208s
sys     0m0.020s

It would be interesting to see how it runs on other machines,
with more cores.

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tbray16_tuple.erl
Type: text/x-erlang
Size: 2490 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/71115e7c/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_tuple.erl
Type: text/x-erlang
Size: 3497 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/71115e7c/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tbray_ets.erl
Type: text/x-erlang
Size: 2622 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/71115e7c/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_ets.erl
Type: text/x-erlang
Size: 3848 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/71115e7c/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tbray_blockread.erl
Type: text/x-erlang
Size: 3308 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/71115e7c/attachment-0004.bin>

From w.a.de.jong@REDACTED  Tue Oct 23 21:35:13 2007
From: w.a.de.jong@REDACTED (Willem de Jong)
Date: Tue, 23 Oct 2007 21:35:13 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>
	<00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
	<Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
Message-ID: <407d9ef80710231235p2133b623lcdc7a097a7b9cb40@mail.gmail.com>

Using erlsom, you can write:

-module(product).
-compile(export_all).

parse(File) ->
  {ok, Model} = erlsom:compile_xsd_file("product.xsd"),
  {ok, Result, _} = erlsom:scan_file(File, Model),
  Result.
Then you can do:
1> erlsom:write_xsd_hrl_file("product.xsd", "product.hrl", []).
2> rr("product.hrl").
3> product:parse("export.xml").
#'Export'{anyAttribs = [],
          'Product' = #'Product'{anyAttribs = [],
                                 'SKU' = "403276",
                                 'ItemName' = "Trivet",
                                 'CollectionNo' = 0,
                                 'Pages' = 0}}

Very different from the example, but also nice :) And maybe more useful,
depending on what you want to do with it.

You need to provide a schema, of course. I am pasting an example schema for
this XML at the end of this email. Using the schema has the advantage that
the xml document will be validated. Having a schema is useful as well to
document the interface. Even if you don't like XML schema's (I still have
problems with them, even after writing the parser), having a specification
should be a good thing. Isn't this a bit like ASN.1, actually?

If you don't like the approach with the schema, you can also do this:

-module(product_sax).
-compile(export_all).

parse(File) ->
  {ok, Bin}  = file:read_file(File),
  {R, _} = erlsom_sax:parseDocument(binary_to_list(Bin), {s1, []},
                                    fun callback/2),
  lists:reverse(R).

callback({startElement, _, "Product", _, _}, {s1, S}) ->
  {s2, S};
callback({startElement, _, Tag, _, _}, {s2, S}) ->
  {s3, {Tag, S}};
callback({characters, Value}, {s3, {Tag, List}}) ->
  {s2, [{Tag, Value} | List]};
callback({endElement, _, "Product", _}, {_, S}) -> S;
callback(_, S) -> S.
4> product_sax:parse("export.xml").
[{"SKU","403276"},{"ItemName","Trivet"},{"CollectionNo","0"},{"Pages","0"}]

Using a callback and a simple sort of state machine - also nice, and very
efficient.

I have written a new version of the sax parser that can parse a file in
blocks, so that you can use it to parse very big files or streams of data.
At the moment I am doing some final testing, finishing the documentation
etc. Not the kind of work I like, so it is likely to take a while (1 - 2
weeks). The new release also fixes some bugs in the XML Schema related code,
and it has some features that should improve the capabilities to use erlsom
for SOAP.

Regards,
Willem

--------------------------
The schema:

<xsd:schema xmlns:xsd='http://www.w3.org/2001/XMLSchema'>

<xsd:element name='Export'>
  <xsd:complexType>
    <xsd:sequence>
      <xsd:element name = 'Product' type='Product'/>
    </xsd:sequence>
  </xsd:complexType>
</xsd:element>

<xsd:complexType name='Product'>
  <xsd:sequence>
    <xsd:element name='SKU' type='xsd:string'/>
    <xsd:element name='ItemName' type='xsd:string'/>
    <xsd:element name='CollectionNo' type='xsd:integer'/>
    <xsd:element name='Pages' type='xsd:integer'/>
  </xsd:sequence>
</xsd:complexType>

</xsd:schema>



On 10/23/07, Hakan Mattsson <hakan@REDACTED> wrote:
>
> On Tue, 23 Oct 2007, Joel Reymont wrote:
>
> JR> Take a look at the following [1] and try to visualize an
> JR> implementation in Erlang. More thoughts after the example.
> JR>
> JR> The data:
> JR>
> JR> <Export>
> JR>    <Product>
> JR>      <SKU>403276</SKU>
> JR>      <ItemName>Trivet</ItemName>
> JR>      <CollectionNo>0</CollectionNo>
> JR>      <Pages>0</Pages>
> JR>    </Product>
> JR> </Export>
> JR>
> JR> The Ruby hPricot code:
> JR>
> JR> FIELDS = %w[SKU ItemName CollectionNo Pages]
> JR>
> JR> doc = Hpricot.parse(File.read("my.xml"))
> JR> (doc/:product).each do |xml_product|
> JR>    product = Product.new
> JR>    for field in FIELDS
> JR>      product[field] = (xml_product/field.intern).first.innerHTML
> JR>    end
> JR>    product.save
> JR> end
>
> At a first glance your Ruby code looks impressively
> compact.  But the corresponding implementation in
> Erlang is about the same size. What's the point in
> adding some syntactic sugar in order to make it even
> more compact? It is just a matter of taste.
>
>    % cat product.erl
>    -module(product).
>    -compile(export_all).
>    -include_lib("xmerl/include/xmerl.hrl").
>
>     parse(File) ->
>        {#xmlElement{content = Exports}, _} = xmerl_scan:file(File),
>        [{Tag, Val} || #xmlElement{content = Products} <- Exports,
>                       #xmlElement{content = Fields} <- Products,
>                       #xmlText{parents = [{Tag, _} | _], value = Val}  <-
> Fields].
>
>    % erl
>    Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0]
> [kernel-poll:false]
>
>    Eshell V5.5.5  (abort with ^G)
>    1> product:parse("my.xml").
>
> [{'SKU',"403276"},{'ItemName',"Trivet"},{'CollectionNo',"0"},{'Pages',"0"}]
>    2>
>
> /H?kan
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/ae2c6128/attachment.htm>

From erlang@REDACTED  Tue Oct 23 22:01:35 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Tue, 23 Oct 2007 22:01:35 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
Message-ID: <9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>

The point is (or was) that firefox has code to parse virtally any kind of broken
warped incomprehensable html - letting firefox figure out the "meaning" of
deeply crippled and totally incomprehensible html and then scanning the result
(the generated DOM) seems a lot easier than figuring out how to parse
crippled HTML yourself - using other stuff as components to do what they are
good at doesn't seem that crazy to me.

/Joe



On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
>
> On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:
>
> > You could then use Erlang as a coordination language controlling
> > a load of firefoxes on different machines, telling them to go get
> > pages and
> > scrape the pages for data which they send back to Erlang.
>
> This is nuts!!! /With all due respect to Joe/
>
> --
> http://wagerlabs.com
>
>
>
>
>
>


From dave.rafkind@REDACTED  Tue Oct 23 22:15:50 2007
From: dave.rafkind@REDACTED (Dave Rafkind)
Date: Tue, 23 Oct 2007 16:15:50 -0400
Subject: [erlang-questions] ets error with shell startup params
Message-ID: <2ae2b2da0710231315w2d1f43d6w6c805dc98e982029@mail.gmail.com>

Hello list, I'm having some trouble with ETS. I have the following code in
tracker.erl

dave1() ->
  ObjectSet = ets:new(object_set, [set, {keypos, 5}]),
  spawn(tracker, dave2, [[], ObjectSet]).

dave2(ClientList, ObjectSet) ->
  MyList = ets:tab2list(ObjectSet),
  io:format("ObjectSet: ~p~n", [MyList]).


This code works fine if I do

   erl -boot start_sasl -pz ebin -sname dave

and then run tracker:dave1() from the shell.

But if I do

  erl -boot start_sasl -pz ebin -sname dave -s tracker dave1

Then I get this error in the shell on startup:

=ERROR REPORT==== 23-Oct-2007::16:07:11 ===
Error in process <0.46.0> on node 'dave@REDACTED' with exit value:
{badarg,[{ets,match_object,[16,'_']},{ets,tab2list,1},{tracker,dave2,2}]}



What's up with that? Do I have to enforce some ordering of application
startup?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/6a31aadd/attachment.htm>

From dave.rafkind@REDACTED  Tue Oct 23 22:29:33 2007
From: dave.rafkind@REDACTED (Dave Rafkind)
Date: Tue, 23 Oct 2007 16:29:33 -0400
Subject: [erlang-questions] ets error with shell startup params
In-Reply-To: <2ae2b2da0710231315w2d1f43d6w6c805dc98e982029@mail.gmail.com>
References: <2ae2b2da0710231315w2d1f43d6w6c805dc98e982029@mail.gmail.com>
Message-ID: <2ae2b2da0710231329h37951970wf7a6e90cb40df8cf@mail.gmail.com>

Ahh nevermind, this was it.

http://www.erlang.org/pipermail/erlang-questions/2002-June/005098.html

On 10/23/07, Dave Rafkind <dave.rafkind@REDACTED> wrote:
>
> Hello list, I'm having some trouble with ETS. I have the following code in
> tracker.erl
>
> dave1() ->
>   ObjectSet = ets:new(object_set, [set, {keypos, 5}]),
>   spawn(tracker, dave2, [[], ObjectSet]).
>
> dave2(ClientList, ObjectSet) ->
>   MyList = ets:tab2list(ObjectSet),
>   io:format("ObjectSet: ~p~n", [MyList]).
>
>
> This code works fine if I do
>
>    erl -boot start_sasl -pz ebin -sname dave
>
> and then run tracker:dave1() from the shell.
>
> But if I do
>
>   erl -boot start_sasl -pz ebin -sname dave -s tracker dave1
>
> Then I get this error in the shell on startup:
>
> =ERROR REPORT==== 23-Oct-2007::16:07:11 ===
> Error in process <0.46.0> on node 'dave@REDACTED' with exit value:
> {badarg,[{ets,match_object,[16,'_']},{ets,tab2list,1},{tracker,dave2,2}]}
>
>
>
> What's up with that? Do I have to enforce some ordering of application
> startup?
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/937f6cd0/attachment.htm>

From vinoski@REDACTED  Tue Oct 23 22:41:58 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Tue, 23 Oct 2007 16:41:58 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
Message-ID: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>

On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> To summarize my progress on the widefinder problem
> A few days ago I started with Steve Vinoski's tbray16.erl
> As a baseline on my 1.66 GHz dual core Centrino
> laptop, Linux,
> tbray16
> real    0m7.067s
> user    0m12.377s
> sys     0m0.584s


Anders, thanks for collecting and posting these. I've just performed a set
of new timings for all of them, as listed below. For each, I just ran this
command:

time erl -smp -noshell -run <test_case> main o1000k.ap >/dev/null

where "<test_case>" is the name of the tbray test case file. All were
looped ten times, and I took the best timing for each. All tests were done
on my 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM Linux box, in a local
(non-NFS) directory.

My original tbray16 runs in

real    0m3.162s
user    0m16.513s
sys     0m1.762s

I removed the dict used for the shift table,
> and changed the min_heap_size.
> That gave
> real    0m2.713s
> user    0m4.168s
> sys     0m0.412s
>
> (see tbray_tuple.erl and wfbm4_tuple.erl)
> Steve reported that it ran in ~1.9 s on his 8 core server.


What I get for tbray_tuple is:

real    0m2.285s
user    0m8.615s
sys     0m0.988s



> Then I removed the dicts that were used for collecting the
> matches and used ets instead, and got some improvement
> on my dual core laptop.
> real    0m2.220s
> user    0m3.252s
> sys     0m0.344s
>
> (see tbray_ets.erl and wfbm4_ets.erl)
>
> Interestingly Steve reported that it actually performed
> worse on his 8 core server.


The discrepancy seems to be gone. With your new file that you supplied in
your message, the official timing for tbray_ets on the 8-core is:

real    0m1.868s
user    0m7.416s
sys     0m0.509s



> These versions all read the whole file into memory at the start.
> On my laptop that takes ~400ms (when the file is already cached
> in the OS).
>
> So I changed it to read the file in chucks and spawn the worker
> after each chunk is read.
>
> tbray_blockread with 4 processes
> real    0m1.992s
> user    0m3.176s
> sys     0m0.420s
>
> (see tbray_blockread.erl and wfbm4_ets.erl)
>
> Running it in the erlang shell it takes ~1.8s.


Interestingly, some of my earlier attempts tried to overlap block reads and
worker spawning, but the results were always worse, so that's why I went to
reading in the whole file. This blockread approach may very well be The
Ultimate Wide Finder.

Timing for tbray_blockread on the 8-core:

real    0m1.546s
user    0m7.337s
sys     0m0.662s


Just starting and stopping the VM takes
> time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
>
> real    0m1.229s
> user    0m0.208s
> sys     0m0.020s


On the 8-core this takes:

real    0m1.093s
user    0m0.072s
sys     0m0.012s


It would be interesting to see how it runs on other machines,
> with more cores.


Tim Bray is traveling at the moment, but he told me by email that he hopes
to get back to measuring these on the T5120 next week.

thanks,
--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/44a147e2/attachment.htm>

From exta7@REDACTED  Tue Oct 23 22:45:59 2007
From: exta7@REDACTED (Zvi)
Date: Tue, 23 Oct 2007 13:45:59 -0700 (PDT)
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710230713i1d104ea1y1ba2866f89687762@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<9b08084c0710230713i1d104ea1y1ba2866f89687762@mail.gmail.com>
Message-ID: <13373230.post@talk.nabble.com>




Joe Armstrong-2 wrote:
> 
> 3) Do you need to handle streams of xml terms. This is often problematic
>     since nobody can agree on the framing protocol - does each term begin
>     with a new <?xml ...?> header?
> 

There are many applications, which using custom and/or non-conformant XML
processing tools, which emit XML without <?xml ...?> processing instruction,
so it's better to have non-strict mode, which accepts XML like this.


Joe Armstrong-2 wrote:
> 
> 4) Do you want validation. In which case how do you find the DTD/schema -
> do you
>     have to comply with OASIS catalogues?
>     Do you want DTD's, Schema, RNC, (or an erlang ad hock grammar)
> 
>     (My solution is to parameterize the parser with a fun F - F(URI)
> is a function
>      that knows how to find the DTD in URI - the OASIS catalogue structure
>      is not something that I want to be concerned with - most
> applications seem to
>      titall ignore this)
> 

Form my expierence DTD is largely ignored today, I can easily live with XML
parser ignoring DTD.
The standard XML Schema language is W3D XSD, which is very usefull,
especially for XML Data Binding and Web Services.


Joe Armstrong-2 wrote:
> 
> 5) Do you want the parser to try an correct errors and recover, or
> bail out early.
> 

the XML parser, should be strict parser (maybe except only leading
<?xml...?> PI.
The HTML and RDF/RSS parsers can be more forgiving, b/c of legacy of
millions of bad HTML.
FOr HTML the solution is Tidy, maybe we need Erlang Tidy port or just call
it via os:cmd .


Joe Armstrong-2 wrote:
> 
> 6) Do you want unicode support? or just ASCII.
> 
if you writing parser for XML configuration files, then ASCII is fine, but
let's agree: it's stupid, it's much easier to store configuration in Plain
Old Erlang Term (POET) format and just file:consult.

For the webapp and web syndication domain the UTF-8 encoding is a MUST. Most
RSS files are UTF-8.


Joe Armstrong-2 wrote:
> 
> 7) Does the data come from files, sockets, binaries?
> 

In most generic case the data comes in chunks of UTF-8 encoded binaries.


Joe Armstrong-2 wrote:
> 
> 8) Do you want strict or lazy parsing. If you don't look an attribute
> you might like to
>     defer parsing the content until you actually need it.
> 

I think a lazy parser just give an illusion of fast processing, it's still
reads the raw XML into memory.
The MSXML DOM native parser was lazy parser from the begining and it was
showing super-fast bechmarks in just "parsing" (read "opening") of XML
files, but when actual processing of XML nodes was involved it was much
slower, than non-lazy Java parsers. Whoever want to process just parts of
XML document, better use SAX, than lazy tree building parser.


Joe Armstrong-2 wrote:
> 
> 9) Do you want to check all ids and idrefs and correctly handle
> NOTATION's  and so on
>     i.e. all the weird things in the XML spec that 99.9% of
> programmers have never used.
> 

no, the only important parts are: namespaces, processing instructions and
CDATA


Joe Armstrong-2 wrote:
> 
> 10) In the case on XML without a DTD do you want a heuristic to throw away
>       non-significant white space (I often use a simple heuristic if all
> the
>       PCDATA children of a tag are all white space then throw all this
> white space away)
> 

good enough for me


Joe Armstrong-2 wrote:
> 
> ...
> 
> It's very difficult to write a parser that correctly handles *all* of
> these and is fast, small etc.
> 
> I have made a set of compromises and a toolkit that provides the
> following.
> 
>    1) A tag level interface to the system
>        this has a file type interface.
> 
>         Pid = open_xml_token_stream(Descriptor)
> 
>       makes a re-entrant token scanner
> 
>        get_next_token(Pid) -> Token | eof
> 
>       A lot of things can be done with this alone, for example a SAX
> like processor
> 
>     2) A simple parser this takes a token stream and parses it just
> checking for
>         well formed-ness.
> 
>     3) A validator that runs on the output of 2 - this only
> understands DTDs (not schemas,
>        or rnc)
> 
>     There are also diverse routines to parse files etc, based on these.
> 
>     I've also written an XSLT type thing that takes the output of 2)
> or 3) and transforms
> it.
> 
>      This is just ASCII.
> 
>      I've talked to a lot of people about XML - most people (the majority)
> want
> something to parse a small ASCII file containing a single XML data
> structure.
> 
>       The data structure is well formed - there is no DTD - and they don't
> care
> about integrity constraints on the attributes- They don't care about
> entity expansion,
> NOTATIONs CDATA etc.
> 

There are two major types of XML:

1. Document-oriented (also Unstructured)
2. Data-oriented (also Structured)

To support document-oriented XML, you need generic XML parser with
namespaces, CDATA, PI, UTF-8 and XPath/XQuery support.

For data-oriented XML, you do not need generic parser, but XML Data Binding
tool, like XSD to Erlang compiler (xsd2erl) (like XMLBeans for Java, or
RogueWave LIEF for C++, etc.), i.e. you giving employee.xsd and it's
compiled into employee.erl module, which you then use in your application to
parse and generate XML instances, conforming to employee.xsd schema.


Joe Armstrong-2 wrote:
> 
> 
>      The kind of Ruby code shown in an earlier posting is easy given a
> simple parse tree
> 
>      My experimental parser turns an XML data structure in a
> 
>       @type xml() = {node,Line,Tag,attrs(),[xml()]} |
> {raw,Ln,AllBlack:bool(), bin()}
> 
>      It's easy to write a fold function that applies a Fun to each node
> 
> fold_over_nodes(Fun, Env, {node, _, _, _, C} = Node) ->
>     Env1 = Fun(Node, Env),
>     fold_over_nodes(Fun, Env1, C);
> fold_over_nodes(Fun, Env, {raw,_,_,_} = Raw) ->
>     Fun(Raw, Env);
> fold_over_nodes(Fun, Env, [H|T]) ->
>     Env1 = fold_over_nodes(Fun, Env, H),
>     fold_over_nodes(Fun, Env1, T);
> fold_over_nodes(Fun, Env, []) ->
>     Env.
> 
>    (like foldl) - this can be used to extract tags
> 
>      F = fun({node,_,Stag,_,C}=N, E) ->
>             case member(Stag, [a,b,c]) of
>                 true -> [N|E];
>                 false -> E
>             end,
>      Tags = fold_over_nodes(Tree, [], F)
> 

I think it should be:
    Tags = fold_over_nodes(F, [], Tree).


Joe Armstrong-2 wrote:
> 
> 
> For very simple applications I could put together a parser that does
> the following:
> 
>    1) Small files only.
>    2) no DTDs or grammar checking
>    3) white space normalisation according to the following.
>        If a tag has MIXED content (ie tags and PCDATA) and all the
> PCDATA bits are all
>        blank then remove all the PCDATA
>    4) ASCII
>    5) simple "obvious" parse tree {tag,Line,Name,[Attr], [Children]}
>        Attr is a sorted [{Key,Val}] list
>    6) Simple SAX library, foldtree functions, find object in tree (like
> Xpath
>        only in Erlang)
> 
> I have a 6-pack of parsers than almost do this - they are all specialsed
> in
> different ways for infinite streams and so on ...
> 
> I'm not sure if a general purpose toolkit (that allows you to build
> the above) or a set
> of completely different parsers with different properties is desirable.
> 
> /Joe Armstrong
> 

Zvi

-- 
View this message in context: http://www.nabble.com/Rant%3A-I-hate-parsing-XML-with-Erlang-tf4676760.html#a13373230
Sent from the Erlang Questions mailing list archive at Nabble.com.



From exta7@REDACTED  Tue Oct 23 22:56:20 2007
From: exta7@REDACTED (Zvi)
Date: Tue, 23 Oct 2007 13:56:20 -0700 (PDT)
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
	<9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
Message-ID: <13373590.post@talk.nabble.com>


using IE on Windows or Firefox on Linux is actually the best way to implement
Web Automation, i.e. bot logging into website/webapp, clicking links and
buttons, etc. This way not only bad-formed HTML, but virtually any Web
technology, like cookies, javascript, AJAX, Flash, plugins,Java applets, can
be supported. 
   The only problems with this approach:
1. It requires much more resources (i.e more heavyweight, than jsut HTML
parsing).
2. When running multiple firefox instances on the same node, there are canbe
security problems.
3. In server environment it should be possible to run Firefox in headless
mode (i.e. without X).

Zvi


Joe Armstrong-2 wrote:
> 
> The point is (or was) that firefox has code to parse virtally any kind of
> broken
> warped incomprehensable html - letting firefox figure out the "meaning" of
> deeply crippled and totally incomprehensible html and then scanning the
> result
> (the generated DOM) seems a lot easier than figuring out how to parse
> crippled HTML yourself - using other stuff as components to do what they
> are
> good at doesn't seem that crazy to me.
> 
> /Joe
> 
> 
> 
> On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
>>
>> On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:
>>
>> > You could then use Erlang as a coordination language controlling
>> > a load of firefoxes on different machines, telling them to go get
>> > pages and
>> > scrape the pages for data which they send back to Erlang.
>>
>> This is nuts!!! /With all due respect to Joe/
>>
>> --
>> http://wagerlabs.com
>>
>>
>>
>>
>>
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 

-- 
View this message in context: http://www.nabble.com/Rant%3A-I-hate-parsing-XML-with-Erlang-tf4676760.html#a13373590
Sent from the Erlang Questions mailing list archive at Nabble.com.



From anders.nygren@REDACTED  Tue Oct 23 23:12:41 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 23 Oct 2007 16:12:41 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
Message-ID: <d3520a2f0710231412y48ab6ccjbf61a54d131e1dfc@mail.gmail.com>

On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> To summarize my progress on the widefinder problem
> A few days ago I started with Steve Vinoski's tbray16.erl
> As a baseline on my 1.66 GHz dual core Centrino
> laptop, Linux,
> tbray16
> real    0m7.067s
> user    0m12.377s
> sys     0m0.584s
>
> I removed the dict used for the shift table,
> and changed the min_heap_size.
> That gave
> real    0m2.713s
> user    0m4.168s
> sys     0m0.412s
>
> (see tbray_tuple.erl and wfbm4_tuple.erl)
> Steve reported that it ran in ~1.9 s on his 8 core server.
>
> Then I removed the dicts that were used for collecting the
> matches and used ets instead, and got some improvement
> on my dual core laptop.
> real    0m2.220s
> user    0m3.252s
> sys     0m0.344s
>
> (see tbray_ets.erl and wfbm4_ets.erl)
>
> Interestingly Steve reported that it actually performed
> worse on his 8 core server.
>
> These versions all read the whole file into memory at the start.
> On my laptop that takes ~400ms (when the file is already cached
> in the OS).
>
> So I changed it to read the file in chucks and spawn the worker
> after each chunk is read.
>
> tbray_blockread with 4 processes
> real    0m1.992s
> user    0m3.176s
> sys     0m0.420s
>
> (see tbray_blockread.erl and wfbm4_ets.erl)
>
> Running it in the erlang shell it takes ~1.8s.
>

In the last email I mentioned that
" Just starting and stopping the VM takes
 time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop

 real    0m1.229s
 user    0m0.208s
 sys     0m0.020s"

But I just realized that a more useful measure for basic
startup and shutdown is

time erl -pa ../../bfile/ebin/ -smp -noshell -run erlang halt

real    0m0.201s
user    0m0.180s
sys     0m0.016s

/Anders


From dmercer@REDACTED  Tue Oct 23 23:25:36 2007
From: dmercer@REDACTED (David Mercer)
Date: Tue, 23 Oct 2007 16:25:36 -0500
Subject: [erlang-questions] Interfacing Erlang With IBM's Websphere MQ
In-Reply-To: <e9447a400710221123lf324babod94b4209ac67f797@mail.gmail.com>
References: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>
	<e9447a400710221123lf324babod94b4209ac67f797@mail.gmail.com>
Message-ID: <00da01c815bb$44a4fc70$891ea8c0@SSI.CORP>

On Monday, October 22, 2007, Patrick Logan wrote:
> Heavier mechanisms for Erlang would be one of the several Java
> interfaces (JMS or MQ-specific), then go through Erlang's Java
> library.
> 
> Or the MQ C/C++ interfaces.

Is that the same as developing a Java or C/C++ port, or is this something
different?

Thank-you.

David



From toby@REDACTED  Wed Oct 24 00:25:33 2007
From: toby@REDACTED (Toby Thain)
Date: Tue, 23 Oct 2007 19:25:33 -0300
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
Message-ID: <A2CD578A-89A4-46E2-8D81-BCAA914FFABE@smartgames.ca>


On 23-Oct-07, at 12:09 PM, Joe Armstrong wrote:

> I've seen some work on parsing badly formed HTML.
>
> If I remember rightly you keep a stack of the currently open tags
> then stacks for things like <font> <b> <i> tags etc. So you end up  
> with
> several small stacks. Each new open or close tag pushes or pops
> things onto these stacks.
>
> When you hit raw data you pattern match over the stacks to figure out
> what to do.
>
> As an aside it occurred to me that mozilla is probably pretty good at
> sceen scraping

Not to mention lynx.

--Toby

> (or whatever it's called) - so it should be possible to write
> a Firefox extension to do this that talks through a socket to Erlang.
> <somebody told me this was easy, but they obviously knew more than  
> I do>
>
> You could then use Erlang as a coordination language controlling
> a load of firefoxes on different machines, telling them to go get  
> pages and
> scrape the pages for data which they send back to Erlang.
>
> if we could use firefox as a component then we could avoid reinventing
> the wheel (again)
>
> /Joe
>
>
> On 10/23/07, Kevin A. Smith <kevin@REDACTED> wrote:
>> Possibly. My understanding was that it still required well-formed
>> documents to function. A lot of feeds feature varying amounts of
>> "well-formedness", sadly.
>>
>> --Kevin
>> On Oct 23, 2007, at 10:01 AM, Joel Reymont wrote:
>>
>>>
>>> On Oct 23, 2007, at 2:46 PM, Kevin A. Smith wrote:
>>>
>>>> FWIW, I tried writing a very permissive feedparser but lost
>>>> interest partially due to the ugliness of Erlang's XML parsing  
>>>> APIs.
>>>
>>> Running yaws_html:parse/1 on a sample RSS feed works just fine. I
>>> suspect you can't get anymore permissive than that.
>>>
>>> --
>>> http://wagerlabs.com
>>>
>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From vinoski@REDACTED  Tue Oct 23 23:41:14 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Tue, 23 Oct 2007 17:41:14 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710231412y48ab6ccjbf61a54d131e1dfc@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710231412y48ab6ccjbf61a54d131e1dfc@mail.gmail.com>
Message-ID: <65b2728e0710231441k71515277t11dff2ed75383981@mail.gmail.com>

On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > To summarize my progress on the widefinder problem
> > A few days ago I started with Steve Vinoski's tbray16.erl
> > As a baseline on my 1.66 GHz dual core Centrino
> > laptop, Linux,
> > tbray16
> > real    0m7.067s
> > user    0m12.377s
> > sys     0m0.584s
> >
> > I removed the dict used for the shift table,
> > and changed the min_heap_size.
> > That gave
> > real    0m2.713s
> > user    0m4.168s
> > sys     0m0.412s
> >
> > (see tbray_tuple.erl and wfbm4_tuple.erl)
> > Steve reported that it ran in ~1.9 s on his 8 core server.
> >
> > Then I removed the dicts that were used for collecting the
> > matches and used ets instead, and got some improvement
> > on my dual core laptop.
> > real    0m2.220s
> > user    0m3.252s
> > sys     0m0.344s
> >
> > (see tbray_ets.erl and wfbm4_ets.erl)
> >
> > Interestingly Steve reported that it actually performed
> > worse on his 8 core server.
> >
> > These versions all read the whole file into memory at the start.
> > On my laptop that takes ~400ms (when the file is already cached
> > in the OS).
> >
> > So I changed it to read the file in chucks and spawn the worker
> > after each chunk is read.
> >
> > tbray_blockread with 4 processes
> > real    0m1.992s
> > user    0m3.176s
> > sys     0m0.420s
> >
> > (see tbray_blockread.erl and wfbm4_ets.erl)
> >
> > Running it in the erlang shell it takes ~1.8s.
> >
>
> In the last email I mentioned that
> " Just starting and stopping the VM takes
> time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
>
> real    0m1.229s
> user    0m0.208s
> sys     0m0.020s"
>
> But I just realized that a more useful measure for basic
> startup and shutdown is
>
> time erl -pa ../../bfile/ebin/ -smp -noshell -run erlang halt
>
> real    0m0.201s
> user    0m0.180s
> sys     0m0.016s


Hmm, good point. On the 8-core:

real    0m0.090s
user    0m0.061s
sys     0m0.012s


--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/53082c69/attachment.htm>

From yinso.chen@REDACTED  Tue Oct 23 23:58:03 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 23 Oct 2007 14:58:03 -0700
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
	<9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
Message-ID: <779bf2730710231458u11acda67t31464bd5dc2c330d@mail.gmail.com>

Agreed - utilizing firefox or IE will further allow you to handle javascript
generated DOMs much more easily then having to write a javascript parser
yourself, which will enable handling of a much larger sets of pages.

But is this *easy* to do within Erlang?

On 10/23/07, Joe Armstrong <erlang@REDACTED> wrote:
>
> The point is (or was) that firefox has code to parse virtally any kind of
> broken
> warped incomprehensable html - letting firefox figure out the "meaning" of
> deeply crippled and totally incomprehensible html and then scanning the
> result
> (the generated DOM) seems a lot easier than figuring out how to parse
> crippled HTML yourself - using other stuff as components to do what they
> are
> good at doesn't seem that crazy to me.
>
> /Joe
>
>
>
> On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
> >
> > On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:
> >
> > > You could then use Erlang as a coordination language controlling
> > > a load of firefoxes on different machines, telling them to go get
> > > pages and
> > > scrape the pages for data which they send back to Erlang.
> >
> > This is nuts!!! /With all due respect to Joe/
> >
> > --
> > http://wagerlabs.com
> >
> >
> >
> >
> >
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/22b920aa/attachment.htm>

From yinso.chen@REDACTED  Wed Oct 24 00:14:37 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 23 Oct 2007 15:14:37 -0700
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <779bf2730710231458u11acda67t31464bd5dc2c330d@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
	<9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
	<779bf2730710231458u11acda67t31464bd5dc2c330d@mail.gmail.com>
Message-ID: <779bf2730710231514t5ea5663ax6886d0aa937cb6f4@mail.gmail.com>

Of course - others on the internet have thought of the same issues and have
blogged about it...

http://emacspeak.blogspot.com/2007/06/firebox-put-fox-in-box.html

Which can run firefox headless + having an REPL with firefox @ the same
time... the rest would just be figuring out the vocabs to talk to firefox
over socket...

And apparently you can do all that in emacs -
http://emacspeak.googlecode.com/svn/trunk/lisp/emacspeak-moz.el.

And the engine behind the REPL - http://beta.hyperstruct.net/projects/mozlab.


On 10/23/07, YC <yinso.chen@REDACTED> wrote:
>
> Agreed - utilizing firefox or IE will further allow you to handle
> javascript generated DOMs much more easily then having to write a javascript
> parser yourself, which will enable handling of a much larger sets of pages.
>
> But is this *easy* to do within Erlang?
>
> On 10/23/07, Joe Armstrong <erlang@REDACTED> wrote:
> >
> > The point is (or was) that firefox has code to parse virtally any kind
> > of broken
> > warped incomprehensable html - letting firefox figure out the "meaning"
> > of
> > deeply crippled and totally incomprehensible html and then scanning the
> > result
> > (the generated DOM) seems a lot easier than figuring out how to parse
> > crippled HTML yourself - using other stuff as components to do what they
> > are
> > good at doesn't seem that crazy to me.
> >
> > /Joe
> >
> >
> >
> > On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
> > >
> > > On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:
> > >
> > > > You could then use Erlang as a coordination language controlling
> > > > a load of firefoxes on different machines, telling them to go get
> > > > pages and
> > > > scrape the pages for data which they send back to Erlang.
> > >
> > > This is nuts!!! /With all due respect to Joe/
> > >
> > > --
> > > http://wagerlabs.com
> > >
> > >
> > >
> > >
> > >
> > >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/dc26afc7/attachment.htm>

From anders.nygren@REDACTED  Wed Oct 24 00:11:12 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 23 Oct 2007 17:11:12 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
Message-ID: <d3520a2f0710231511s118113a2sb1a3a84c42574c6c@mail.gmail.com>

On 10/23/07, Steve Vinoski <vinoski@REDACTED> wrote:
> On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > To summarize my progress on the widefinder problem
> > A few days ago I started with Steve Vinoski's tbray16.erl
> > As a baseline on my 1.66 GHz dual core Centrino
> > laptop, Linux,
> > tbray16
> > real    0m7.067s
> > user     0m12.377s
> > sys     0m0.584s
>
> Anders, thanks for collecting and posting these. I've just performed a set
> of new timings for all of them, as listed below. For each, I just ran this
> command:
>
> time erl -smp -noshell -run <test_case> main o1000k.ap >/dev/null
>
> where "<test_case>" is the name of the tbray test case file. All were looped
> ten times, and I took the best timing for each. All tests were done on my
> 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM Linux box, in a local
> (non-NFS) directory.
>

I don't keep track of the finer details of different CPUs, but I have
a vague memory of that the 8 core Xeon is really 2 4 core CPUs
on one chip, is that correct?

The reason I am asking is that I can not figure out why Your
measurements have shorter real times than mine, but more
than twice the user time.

Also it does not seems to scale so well up to 8 cores.
Steve's best time is 0m1.546s an mine was 0m1.992s.


Steve, can You also do some tests on tbray_blockread using
different numbers of worker processes. Since smaller block
size means that we start using all the cores earlier.

> My original tbray16 runs in
>
>
> real    0m3.162s
> user    0m16.513s
> sys     0m1.762s
> > I removed the dict used for the shift table,
> > and changed the min_heap_size.
> > That gave
> > real    0m2.713s
> > user    0m4.168s
> > sys     0m0.412s
> >
> > (see tbray_tuple.erl and wfbm4_tuple.erl)
> > Steve reported that it ran in ~1.9 s on his 8 core server.
>
>
> What I get for tbray_tuple is:
>
> real    0m2.285s
> user    0m8.615s
> sys     0m0.988s
>
>
> > Then I removed the dicts that were used for collecting the
> > matches and used ets instead, and got some improvement
> > on my dual core laptop.
> > real    0m2.220s
> > user    0m3.252s
> > sys     0m0.344s
> >
> > (see tbray_ets.erl and wfbm4_ets.erl)
> >
> > Interestingly Steve reported that it actually performed
> > worse on his 8 core server.
>
> The discrepancy seems to be gone. With your new file that you supplied in
> your message, the official timing for tbray_ets on the 8-core is:
>
>
> real    0m1.868s
> user    0m7.416s
> sys     0m0.509s
>
>
> > These versions all read the whole file into memory at the start.
> > On my laptop that takes ~400ms (when the file is already cached
> > in the OS).
> >
> > So I changed it to read the file in chucks and spawn the worker
> > after each chunk is read.
> >
> > tbray_blockread with 4 processes
> > real    0m1.992s
> > user    0m3.176s
> > sys     0m0.420s
> >
> > (see tbray_blockread.erl and wfbm4_ets.erl)
> >
> > Running it in the erlang shell it takes ~1.8s.
>
>
> Interestingly, some of my earlier attempts tried to overlap block reads and
> worker spawning, but the results were always worse, so that's why I went to
> reading in the whole file. This blockread approach may very well be The
> Ultimate Wide Finder.
>
> Timing for tbray_blockread on the 8-core:
>
> real    0m1.546s
> user    0m7.337s
> sys     0m0.662s
>
>
> > Just starting and stopping the VM takes
> > time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
> >
> > real    0m1.229s
> > user    0m0.208s
> > sys     0m0.020s
>
> On the 8-core this takes:
>
> real     0m1.093s
> user    0m0.072s
> sys     0m0.012s
>
> > It would be interesting to see how it runs on other machines,
> > with more cores.
>
> Tim Bray is traveling at the moment, but he told me by email that he hopes
> to get back to measuring these on the T5120 next week.
>
> thanks,
> --steve


From vinoski@REDACTED  Wed Oct 24 00:55:07 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Tue, 23 Oct 2007 18:55:07 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710231511s118113a2sb1a3a84c42574c6c@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
	<d3520a2f0710231511s118113a2sb1a3a84c42574c6c@mail.gmail.com>
Message-ID: <65b2728e0710231555tc0503d1obe81942dc9ecccd0@mail.gmail.com>

On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> On 10/23/07, Steve Vinoski <vinoski@REDACTED> wrote:
> > On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > > To summarize my progress on the widefinder problem
> > > A few days ago I started with Steve Vinoski's tbray16.erl
> > > As a baseline on my 1.66 GHz dual core Centrino
> > > laptop, Linux,
> > > tbray16
> > > real    0m7.067s
> > > user     0m12.377s
> > > sys     0m0.584s
> >
> > Anders, thanks for collecting and posting these. I've just performed a
> set
> > of new timings for all of them, as listed below. For each, I just ran
> this
> > command:
> >
> > time erl -smp -noshell -run <test_case> main o1000k.ap >/dev/null
> >
> > where "<test_case>" is the name of the tbray test case file. All were
> looped
> > ten times, and I took the best timing for each. All tests were done on
> my
> > 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM Linux box, in a local
> > (non-NFS) directory.
> >
>
> I don't keep track of the finer details of different CPUs, but I have
> a vague memory of that the 8 core Xeon is really 2 4 core CPUs
> on one chip, is that correct?


Yes, I believe so.

The reason I am asking is that I can not figure out why Your
> measurements have shorter real times than mine, but more
> than twice the user time.


It's because the user time includes CPU time on all the cores. More cores,
and more things happening on those cores, means more CPU time and thus more
user time. Tim saw the same phenomenon on his T5120 and blogged about it
here:

<http://www.tbray.org/ongoing/When/200x/2007/10/09/Niagara-2-T2-T5120>

Also it does not seems to scale so well up to 8 cores.
> Steve's best time is 0m1.546s an mine was 0m1.992s.


The default settings in the code are probably not ideal for the 8-core box.

Steve, can You also do some tests on tbray_blockread using
> different numbers of worker processes. Since smaller block
> size means that we start using all the cores earlier.


I ran a series of tests of different block sizes, and I found that for the 8
core, dividing the file into 1024 chunks (for this file, this means a block
size of 230606 bytes) produced the best time:

real    0m1.103s
user    0m6.651s
sys     0m0.492s

Which is pretty darn fast. :-) Smaller chunk sizes are slower probably
because there's more result collecting and merging to do, while larger chunk
sizes are slower because parallelism is reduced.

I can't wait to see this thing run on Tim's T5120.

BTW, I got a comment on my blog today from someone who essentially said I
was making Erlang look bad by applying it to a problem for which it's not a
good fit. My response was that I didn't agree; Tim's original goal was to
maximize the use of a multicore system for solving the Wide Finder, and
Erlang now does that better than anything else I've seen so far. Does anyone
in the Erlang community agree with the person who made that comment that
this Wide Finder project has made Erlang look bad?

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/5b165529/attachment.htm>

From erlangx@REDACTED  Wed Oct 24 01:00:56 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Tue, 23 Oct 2007 16:00:56 -0700
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <779bf2730710231458u11acda67t31464bd5dc2c330d@mail.gmail.com>
References: <95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>
	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>
	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>
	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>
	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<3F96F5EF-DFD0-4013-84AA-B45FF43C3896@gmail.com>
	<9b08084c0710231301m69ed7160s365fc50239cd603b@mail.gmail.com>
	<779bf2730710231458u11acda67t31464bd5dc2c330d@mail.gmail.com>
Message-ID: <20071023230056.GC6096@delora.autosys.us>

 On MS Windows I use classMechanizeIE.php
 (http://www.cgi-interactive-uk.com/com_functions_php_ie.html) and a
 small PHP script to grab pages by controlling MS Internet Explorer
 through its COM interface.  My Erlang program manages the various
 jobs and parses the resultant text files created, sending alerts as
 needed.

 My preference would have been something built-in to Erlang for the
 COM control but Comet no longer is integral with the distribution
 (I do not know if it would have been suitable for the task, anyway).
 I had to use Internet Explorer as the browser because the environment
 in which I am doing this task will check for valid login when you go
 to the page (that is, somehow the server knows if you are logged in
 to your workstation and requires you to use MS Internet Explorer to
 automatically authenticate when you go to specific pages).  Simply
 using http:request/4 or lynx or telnet would not authenticate
 properly.


~Michael


On Tue, Oct 23, 2007 at 02:58:03PM -0700, YC wrote:
> Agreed - utilizing firefox or IE will further allow you to handle javascript
> generated DOMs much more easily then having to write a javascript parser
> yourself, which will enable handling of a much larger sets of pages.
> 
> But is this *easy* to do within Erlang?
> 
> On 10/23/07, Joe Armstrong <erlang@REDACTED> wrote:
> 
>     The point is (or was) that firefox has code to parse virtally any kind of
>     broken
>     warped incomprehensable html - letting firefox figure out the "meaning" of
>     deeply crippled and totally incomprehensible html and then scanning the
>     result
>     (the generated DOM) seems a lot easier than figuring out how to parse
>     crippled HTML yourself - using other stuff as components to do what they
>     are
>     good at doesn't seem that crazy to me.
> 
>     /Joe
> 
> 
> 
>     On 10/23/07, Joel Reymont <joelr1@REDACTED> wrote:
>     >
>     > On Oct 23, 2007, at 4:09 PM, Joe Armstrong wrote:
>     >
>     > > You could then use Erlang as a coordination language controlling
>     > > a load of firefoxes on different machines, telling them to go get
>     > > pages and
>     > > scrape the pages for data which they send back to Erlang.
>     >
>     > This is nuts!!! /With all due respect to Joe/
>     >
>     > --
>     > http://wagerlabs.com
>     >
>     >
>     >
>     >
>     >
>     >
>     _______________________________________________
>     erlang-questions mailing list
>     erlang-questions@REDACTED
>     http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> 
> !DSPAM:52,471e700950982146018883!

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,471e700950982146018883!


-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From patrickdlogan@REDACTED  Wed Oct 24 01:23:47 2007
From: patrickdlogan@REDACTED (Patrick Logan)
Date: Tue, 23 Oct 2007 16:23:47 -0700
Subject: [erlang-questions] Interfacing Erlang With IBM's Websphere MQ
In-Reply-To: <00da01c815bb$44a4fc70$891ea8c0@SSI.CORP>
References: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>
	<e9447a400710221123lf324babod94b4209ac67f797@mail.gmail.com>
	<00da01c815bb$44a4fc70$891ea8c0@SSI.CORP>
Message-ID: <e9447a400710231623h136852afv1c3803ca1a2c5ea2@mail.gmail.com>

> > Heavier mechanisms for Erlang would be one of the several Java
> > interfaces (JMS or MQ-specific), then go through Erlang's Java
> > library.
> >
> > Or the MQ C/C++ interfaces.
>
> Is that the same as developing a Java or C/C++ port, or is this something
> different?

You'd have to wrap whichever approach up into a C or Java port.


From francis.norton@REDACTED  Tue Oct 23 14:01:09 2007
From: francis.norton@REDACTED (Francis Norton)
Date: Tue, 23 Oct 2007 13:01:09 +0100
Subject: [erlang-questions] Design methodology going from Object
	oriented to functional programming?
Message-ID: <24162acd0710230501i1ab100bdu5fd7584aa573d859@mail.gmail.com>

I'm just an Erlang learner, but one thing that led me to Erlang is my
ancient and well thumbed copy of Shlaer and Mellor's "Object lifecycles:
Modeling the world in states" (
http://www.amazon.com/Object-Life-Cycles-Modeling-Computing/dp/0136299407)
which, for such a short book, made a great deal of sense to me and very much
changed the way I saw program design.

It might be relevant to you because it is aimed at developing real-time
systems where each "object" is implemented as a state-machine driven
process, and the messages each process accepts are its "methods".

Francis.

On 10/22/07, Alexander Lamb < alexander.lamb@REDACTED > wrote:
>
> Hello list,
> I am trying to understand what is the design process (intellectual, that
> is) when building a program in Erlang.
>
> Indeed, in the object oriented world, I would start by finding what my
> classes might be and the relationship between them. Gradually I would add
> functions (class or instance methods) to the classes in order to provide
> solid foundations on top of which I can write an application.
>

On 10/23/07, erlang-questions-request@REDACTED <
erlang-questions-request@REDACTED> wrote:
>
> Send erlang-questions mailing list submissions to
>         erlang-questions@REDACTED
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://www.erlang.org/mailman/listinfo/erlang-questions
> or, via email, send a message with subject or body 'help' to
>         erlang-questions-request@REDACTED
>
> You can reach the person managing the list at
>         erlang-questions-owner@REDACTED
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of erlang-questions digest..."
>
>
> Today's Topics:
>
>    1. HiPE compilation option (Serge Aleynikov)
>    2. Re: Fwd: Why single-assignment with non-shared    state?
>       (Richard A. O'Keefe)
>    3. Re: Design methodology going from Object  oriented to
>       functional programming? (Aaron Feng)
>    4. Re: Design methodology going from Object  oriented to
>       functional programming? (YC)
>    5. Re: Design methodology going from Object  oriented to
>       functional programming? (Jay Nelson)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 22 Oct 2007 20:39:24 -0500
> From: Serge Aleynikov <saleyn@REDACTED>
> Subject: [erlang-questions] HiPE compilation option
> To: "Erlang Users' List" <erlang-questions@REDACTED>
> Message-ID: <471D50CC.1030601@REDACTED>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> One of my colleagues asked me how to enable HiPE compilation.  After
> explaining the +native command-line flag I went back to documentation
> and haven't found any reference to it in erlc, c, compile.  I think it
> must be an oversight as HiPE is an important feature that deserves to be
> documented.
>
> Serge
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 23 Oct 2007 15:08:07 +1300
> From: "Richard A. O'Keefe" <ok@REDACTED>
> Subject: Re: [erlang-questions] Fwd: Why single-assignment with
>         non-shared      state?
> To: Erlang Questions <erlang-questions@REDACTED>
> Message-ID: <E787BEFE-6AB4-4727-9ADC-AB3A014C057E@REDACTED>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
> On 22 Oct 2007, at 2:05 am, Tobias Gerdin wrote:
>
> > Well as long as the expressions you match on doesn't change during
> > matching I do not see the problem. There are several prominent
> > languages such as the ML-family, Scala, or even Lisps which feature
> > both pattern matching and assignment.
>
> I know nothing about Scala.
>
> However, I keep both SML and Mlton on my machines (hmm, I think I may
> have MoscowML somewhere as well).  SML DOES NOT ALLOW PATTERN MATCHING
> ON MUTABLE DATA.  SML's mutable data are "references" (boxes holding a
> single variable) and "arrays".  You cannot look inside either of these
> using pattern matching.
>
> The only "pattern matching" feature standardly available in Lisp is
> destructuring-bind (and its close relatives).  It matches mutable
> cons cells because Lisp doesn't have any other kind.  Reading any
> Scheme or Lisp tutorial, with its dire warnings against changing
> lists that happened to appear as literal data, may provide some idea
> of the trouble mutable data can get you into.
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 22 Oct 2007 22:29:51 -0400
> From: "Aaron Feng" <aaron.feng@REDACTED>
> Subject: Re: [erlang-questions] Design methodology going from Object
>         oriented to functional programming?
> To: "Jay Nelson" <jay@REDACTED>
> Cc: erlang-questions@REDACTED
> Message-ID:
>         <4e3934860710221929y5b0a775bm10a7df93f206003b@REDACTED>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi Jay,
>
> I found the paper very insightful.  I understand the space constraint that
> some of the code sample has to be removed.  Is the complete code sample
> available for download?
>
> Aaron
>
> On 10/22/07, Jay Nelson <jay@REDACTED> wrote:
> >
> > I wrote a paper for the 2004 ICFP on designing structured systems
> > using processes.  You can find it on my website at http://
> > www.duomark.com/erlang/index.html
> >
> > jay
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://www.erlang.org/pipermail/erlang-questions/attachments/20071022/7b3bfb0a/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Mon, 22 Oct 2007 19:41:26 -0700
> From: YC <yinso.chen@REDACTED>
> Subject: Re: [erlang-questions] Design methodology going from Object
>         oriented to functional programming?
> To: "Alexander Lamb" <alexander.lamb@REDACTED>
> Cc: erlang-questions@REDACTED
> Message-ID:
>         <779bf2730710221941y4865b02bo5c842e94ced49c96@REDACTED>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Actually, at the level you are describing, there should be no difference
> between FP and OOP.
>
> While OOP definitely emphasize data and relations, it is not the only
> paradigm that does so - and given your background you should be
> experienced
> with the relational paradigm, which has even heavier emphasis on data and
> relations, but is not OOP.
>
> To leverage your knowledge on relations - you can pretend sql queries are
> functions, i.e. select is a function, update is a function, and insert is
> a
> function.  Then instead of writing
>
> object.insert()
>
> you write
>
> insert (object) - which looks a lot like the sql query insert into table
> ...
>
>
> That's it.  In the Java style OOP the head of the statement is an object,
> and in FP the head of the statement is a function name.  But in either
> case
> you need to model the same world.
>
> An extremely crude way of thinking about FP is that it decouples the data
> from the function (or vice versa, that OOP couples functions and data).
> That means some facilities that you've taken for granted in OOP, such as
> inheritance, polymorphism, etc., will no longer be available.  But FP have
> a
> different approaches to address these problems, and that's where the
> rubber
> meet the road.
>
> Erlang's process model is basically the Actor model - the idea of
> everything
> is an actor feels similar to everything is an object in OOP.  So as others
> have alluded to instead of thinking in objects you can think in processes.
> But Actor model is independent of FP or OOP, so you would still have to
> get
> used to the FP part in Erlang.
>
> You might want to check out
> http://www.math.chalmers.se/~rjmh/Papers/whyfp.html
> <http://www.math.chalmers.se/%7Erjmh/Papers/whyfp.html>.  A higher level
> introduction is http://www.defmacro.org/ramblings/fp.html.
>
> Cheers,
> yc
>
> On 10/22/07, Alexander Lamb < alexander.lamb@REDACTED > wrote:
> >
> > Hello list,
> > I am trying to understand what is the design process (intellectual, that
> > is) when building a program in Erlang.
> >
> > Indeed, in the object oriented world, I would start by finding what my
> > classes might be and the relationship between them. Gradually I would
> add
> > functions (class or instance methods) to the classes in order to provide
> > solid foundations on top of which I can write an application.
> >
> > For example, I could have PERSON, PROFILE, ROLE, FEATURE, etc... and
> > decide a PROFILE is a collection of FEATURES. A PERSON can have 0 or
> many
> > ROLES. A ROLE is a PROFILE on a given area (a department for example)
> for a
> > given time. I would then add functions such as "give all the active
> roles
> > for the user" or "what features give that profile" or "does the user
> have a
> > given feature for that department".
> > I admit it is more complexe than that, but you get the idea.
> >
> > Obviously, this doesn't seem to be the way to go with Erlang.
> Intuitively,
> > I would start making a list of all the functions which will allow me to
> > interract with my application. In that case I could have "give me all
> users
> > with an active role on that department", etc... Then by implementing
> those
> > high level functions I would split them into pieces by calling smaller
> > simpler functions. The underlying data structure will "just follow" or
> > "appear" naturally.
> >
> > Hence: object oriented design is "data structure and relationships
> first,
> > functions second" and functional design is "functions first, data
> structure
> > second".
> >
> > Am I being over simplistic here. Are there some guidelines as to how one
> > can approach a problem when creating a new program? Especially programs
> > which deal with persistent data, not protocole analysers or socket
> servers!
> >
> > Thanks,
> >
> > Alex
> >   --
> > Alexander Lamb
> > Founding Associate
> > RODANOTECH S?rl
> >
> > 4 ch. de la Tour de Champel
> > 1206 Geneva
> > Switzerland
> >
> > Tel:  022 347 77 37
> > Fax: 022 347 77 38
> >
> > http://www.rodanotech.ch
> >
> >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://www.erlang.org/pipermail/erlang-questions/attachments/20071022/0fc13763/attachment-0001.html
>
> ------------------------------
>
> Message: 5
> Date: Mon, 22 Oct 2007 21:51:44 -0700
> From: Jay Nelson <jay@REDACTED>
> Subject: Re: [erlang-questions] Design methodology going from Object
>         oriented to functional programming?
> To: "Aaron Feng" <aaron.feng@REDACTED>
> Cc: erlang-questions@REDACTED
> Message-ID: <FC68D843-F1BE-4859-AE50-E966D81B4F82@REDACTED>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> I haven't looked at for a couple years.  I don't have the code on the
> web.  I will have to look around and see if I can find it on another
> computer.
>
> jay
>
> On Oct 22, 2007, at 7:29 PM, Aaron Feng wrote:
>
> > Hi Jay,
> >
> > I found the paper very insightful.  I understand the space
> > constraint that some of the code sample has to be removed.  Is the
> > complete code sample available for download?
> >
> > Aaron
> >
>
>
>
> ------------------------------
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
> End of erlang-questions Digest, Vol 5, Issue 69
> ***********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071023/5a800e80/attachment.htm>

From anders.nygren@REDACTED  Wed Oct 24 01:42:28 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 23 Oct 2007 18:42:28 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710231555tc0503d1obe81942dc9ecccd0@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
	<d3520a2f0710231511s118113a2sb1a3a84c42574c6c@mail.gmail.com>
	<65b2728e0710231555tc0503d1obe81942dc9ecccd0@mail.gmail.com>
Message-ID: <d3520a2f0710231642u54d46abeucaeaefd666366d12@mail.gmail.com>

On 10/23/07, Steve Vinoski <vinoski@REDACTED> wrote:
> On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > On 10/23/07, Steve Vinoski <vinoski@REDACTED> wrote:
> > > On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > > > To summarize my progress on the widefinder problem
> > > > A few days ago I started with Steve Vinoski's tbray16.erl
> > > > As a baseline on my 1.66 GHz dual core Centrino
> > > > laptop, Linux,
> > > > tbray16
> > > > real    0m7.067s
> > > > user     0m12.377s
> > > > sys     0m0.584s
> > >
> > > Anders, thanks for collecting and posting these. I've just performed a
> set
> > > of new timings for all of them, as listed below. For each, I just ran
> this
> > > command:
> > >
> > > time erl -smp -noshell -run <test_case> main o1000k.ap >/dev/null
> > >
> > > where "<test_case>" is the name of the tbray test case file. All were
> looped
> > > ten times, and I took the best timing for each. All tests were done on
> my
> > > 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM Linux box, in a local
> > > (non-NFS) directory.
> > >
> >
> > I don't keep track of the finer details of different CPUs, but I have
> > a vague memory of that the 8 core Xeon is really 2 4 core CPUs
> > on one chip, is that correct?
>
> Yes, I believe so.
> > The reason I am asking is that I can not figure out why Your
> > measurements have shorter real times than mine, but more
> > than twice the user time.
>
> It's because the user time includes CPU time on all the cores. More cores,
> and more things happening on those cores, means more CPU time and thus more
> user time. Tim saw the same phenomenon on his T5120 and blogged about it
> here:
>

But user time is supposed to be the time used executing instructions
for the process and its children, i.e. the CPU time used to solve
the task. So the user time should ideally remain constant when
more cores are added, and the real time should ideally be divided
by the number of cores.

But also Tim said
"Further poking dug up the answer: it seems that the hardware doesn't
tell the OS how it's sharing out the cycles among the the threads that
it has runnable at any point in time. So Solaris just credits them
with user CPU time whenever they're in Run state. The results will be
correct when you have up to sixteen threads staying runnable; above
that they get funky. "

So basically You can not trust the user time on the T1 or T2. But I
don't think that also applies on other processors.

> <http://www.tbray.org/ongoing/When/200x/2007/10/09/Niagara-2-T2-T5120>
> > Also it does not seems to scale so well up to 8 cores.
> > Steve's best time is 0m1.546s an mine was 0m1.992s .
>
> The default settings in the code are probably not ideal for the 8-core box.
> > Steve, can You also do some tests on tbray_blockread using
> > different numbers of worker processes. Since smaller block
> > size means that we start using all the cores earlier.
>
>
> I ran a series of tests of different block sizes, and I found that for the 8
> core, dividing the file into 1024 chunks (for this file, this means a block
> size of 230606 bytes) produced the best time:

Yes, I also got the impression that there was a optimum around
200k blocksize. But the sample to sample variation is enough that
I was not sure.

>
>
> real    0m1.103s
> user    0m6.651s
> sys     0m0.492s
>
> Which is pretty darn fast. :-) Smaller chunk sizes are slower probably
> because there's more result collecting and merging to do,

There is no merging or collecting of results in the ets based versions.
I think the slowdown for even smaller blocks is because of the
scheduling of, and switching between, all the worker processes.


>while larger chunk
> sizes are slower because parallelism is reduced.
>
> I can't wait to see this thing run on Tim's T5120.
>
> BTW, I got a comment on my blog today from someone who essentially said I
> was making Erlang look bad by applying it to a problem for which it's not a
> good fit. My response was that I didn't agree; Tim's original goal was to
> maximize the use of a multicore system for solving the Wide Finder, and
> Erlang now does that better than anything else I've seen so far. Does anyone
> in the Erlang community agree with the person who made that comment that
> this Wide Finder project has made Erlang look bad?

I think Tim was unnecessarily strong in his initial comments.
He did a naive beginner solution and when it performed badly he
said the Erlang sucks, (more or less). Instead of asking how to
improve it.

It has been an interesting exercise, and quite useful for me since
I am currently looking at a system that needs to process ~ 1 Tera
byte of log files per day. Fortunately they are BER coded :)

/Anders


From joelr1@REDACTED  Wed Oct 24 02:35:58 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Wed, 24 Oct 2007 01:35:58 +0100
Subject: [erlang-questions] Natural language processing
Message-ID: <05EFAA97-A790-4014-8F9C-9932EEA8B6C7@gmail.com>

Has anyone done natural language processing in Erlang?

Any suggestions on how to start?

I have a book about NLP in Lisp and some other books on statistical  
approaches to NLP.

I'm lazy, though, so I'm just looking for shortcuts.

	Thanks, Joel

--
http://wagerlabs.com







From jeffm@REDACTED  Wed Oct 24 05:40:47 2007
From: jeffm@REDACTED (jm)
Date: Wed, 24 Oct 2007 13:40:47 +1000
Subject: [erlang-questions] turning up debugging on ports
Message-ID: <471EBEBF.4090304@ghostgun.com>

Is there an option to erl which will turn up the debugging so I can see 
what erlang is doing in relation to a port module I'm attempting to 
debug. I tried +v (verbose) but it doesn't appear to help. In particlar 
I wish to find out what the node is doing just before it closes the 
port. Any recommended reading on debugging port would also be welcomed.


Jeff.


From jeffm@REDACTED  Wed Oct 24 05:40:42 2007
From: jeffm@REDACTED (jm)
Date: Wed, 24 Oct 2007 13:40:42 +1000
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <471C3195.6090602@ghostgun.com>
References: <200710201902.l9KJ2cPw094706@pluto.hedeland.org>
	<471C3195.6090602@ghostgun.com>
Message-ID: <471EBEBA.6030207@ghostgun.com>

jm wrote:
> Per Hedeland wrote:
>> jm <jeffm@REDACTED> wrote:
>>> it appears somthing is causing erlang to close stdin to the port,
>> Yes, that was pretty clear from your earlier results.
> 
> Just had to prove it.

In continuing to debug this port program I recompile with debug enabled 
(-g) and attached gdb to the running process which resulted in it 
writting to a broken pipe as shown

205             while (size < len) {
(gdb) n
207               event = (struct inotify_event *) &buf[i];
(gdb) n
208               size += EVENT_SIZE + event->len;
(gdb) n
211               fprintf(stderr, "encoding event wd=%d, mask=%d, 
cookie=%d, len=%d, name=%s\r\n",
(gdb) n
214               if (ei_x_new_with_version(&result) ||
(gdb) n
216               ei_x_encode_atom(&result, "event"); 
       /* element 1 */
(gdb) n
217               ei_x_encode_ulong(&result, event->wd); 
       /* element 2 */
(gdb) print result
$1 = {buff = 0x804e018 "\203h\005d", buffsz = 203, index = 11}
(gdb) n
219               ei_x_encode_atom(&result, "mask");
(gdb) n
220               ei_x_encode_ulong(&result, event->cookie); 
       /* element 4 */
(gdb) n
221               ei_x_encode_string_len(&result, event->name, 
event->len);   /* element 5 */
(gdb) n
222               fprintf(stderr, "note_read send event\r\n");
(gdb) n
223               write_cmd(&result);
(gdb) n

Program received signal SIGPIPE, Broken pipe.
0xb7e8dbde in write () from /lib/libc.so.6
(gdb)



Jeff.


From w.a.de.jong@REDACTED  Wed Oct 24 08:14:21 2007
From: w.a.de.jong@REDACTED (Willem de Jong)
Date: Wed, 24 Oct 2007 08:14:21 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <13373230.post@talk.nabble.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<471DE3F8.3080903@volny.cz>
	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>
	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>
	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>
	<9b08084c0710230713i1d104ea1y1ba2866f89687762@mail.gmail.com>
	<13373230.post@talk.nabble.com>
Message-ID: <407d9ef80710232314v6ba4ec85x44cb25654c375262@mail.gmail.com>

On 10/23/07, Zvi <exta7@REDACTED> wrote:
>
> There are two major types of XML:
>
> 1. Document-oriented (also Unstructured)
> 2. Data-oriented (also Structured)
>
> To support document-oriented XML, you need generic XML parser with
> namespaces, CDATA, PI, UTF-8 and XPath/XQuery support.
>
> For data-oriented XML, you do not need generic parser, but XML Data
> Binding
> tool, like XSD to Erlang compiler (xsd2erl) (like XMLBeans for Java, or
> RogueWave LIEF for C++, etc.), i.e. you giving employee.xsd and it's
> compiled into employee.erl module, which you then use in your application
> to
> parse and generate XML instances, conforming to employee.xsd schema.


Erlsom is an XML Data Binding Tool written in Erlang.

It doesn't generate code (except for a header file with record definitions),
but the effect is the same: When you parse employee.xml, it uses (a
'compiled' version of) employee.xsd to translate it to a structure of
records that correspond to the types defined in the XSD. Or the other way
around: given such a structure, it can generate the XML.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/d4930217/attachment.htm>

From ulf@REDACTED  Wed Oct 24 10:00:51 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Wed, 24 Oct 2007 10:00:51 +0200
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>
	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>
	<00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
	<Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
Message-ID: <8209f740710240100o2bd8d7cay8fd348fe6f90a52@mail.gmail.com>

2007/10/23, Hakan Mattsson <hakan@REDACTED>:
> On Tue, 23 Oct 2007, Joel Reymont wrote:
>
>
> At a first glance your Ruby code looks impressively
> compact.  But the corresponding implementation in
> Erlang is about the same size. What's the point in
> adding some syntactic sugar in order to make it even
> more compact? It is just a matter of taste.
>
>     % cat product.erl
>     -module(product).
>     -compile(export_all).
>     -include_lib("xmerl/include/xmerl.hrl").
>
>      parse(File) ->
>         {#xmlElement{content = Exports}, _} = xmerl_scan:file(File),
>         [{Tag, Val} || #xmlElement{content = Products} <- Exports,
>                        #xmlElement{content = Fields} <- Products,
>                        #xmlText{parents = [{Tag, _} | _], value = Val}  <- Fields].
>
>     % erl
>     Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [kernel-poll:false]
>
>     Eshell V5.5.5  (abort with ^G)
>     1> product:parse("my.xml").
>     [{'SKU',"403276"},{'ItemName',"Trivet"},{'CollectionNo',"0"},{'Pages',"0"}]
>     2>

There is a function, xmerl_lib:simplify_element(E), which accomplishes the
same thing:

1> {Data,_} = xmerl_scan:string(Str),xmerl_lib:simplify_element(Data).
{'Export',[],
          ["\n\n  ",
           {'Product',[],
                      ["\n\n    ",
                       {'SKU',[],["403276"]},
                       "\n\n    ",
                       {'ItemName',[],["Trivet"]},
                       "\n\n    ",
                       {'CollectionNo',[],["0"]},
                       "\n\n    ",
                       {'Pages',[],["0"]},
                       "\n\n  "]},
           "\n\n"]}

Unfortunately, it's not documented.

BR,
Ulf W


From alexander.lamb@REDACTED  Wed Oct 24 12:00:31 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Wed, 24 Oct 2007 12:00:31 +0200
Subject: [erlang-questions] Erlang processes and state management design
Message-ID: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>

Hello again,

So, I am gradually understanding the design philosophy of an Erlang  
application.
On the one hand, functionnal programming is about having no state.  
However, in the situation of a Web application, you obviously need to  
keep some state otherwise you are back to PhP style (in the sense you  
are recalculating everything at each request).

I went back to Joe's book and read again the chapter on processes.  
And, as someone suggested earlier, I realized the state of an Erlang  
program can actually be kept in a process. So when designing an  
application, you indeed model the processes, then the state of the  
processes, then the messages.

So for example, if I create a process for each user who logs in to a  
program. Upon creation of the process I could fetch all the features  
and worklists the user has, then keep them in his process until he  
logs out (or after a given long timeout).

I can do that for every task which needs to keep a state between  
requests.

Therefore, I have a question: where is it usuall or reasonnable to  
keep the state of a process?

1) in the parameters of the function we spawn
2) in the process dictionnary
3) in an ETS structure
4) in a in-memory mnesia structure

I suppose it really depends on the amount of data and how it will be  
modified (and if we want to keep persistent some session data after  
the process is killed).

Tell me if I am wrong, but the first two solutions will put data on  
the process stack and will be destroyed when the process is killed.  
The two other solutions will have to be cleaned out when the process  
is killed.

Are there some guidelines?

For a user process, I might have 30 or 40 features in a list, some  
values about his account, etc...
For a process managing (for example) browsing patients in a list, I  
will have the list of patients with the visits for the currently  
selected patient, etc...

Thanks for any hints!

Alex
--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/c055d83e/attachment.htm>

From chandrashekhar.mullaparthi@REDACTED  Wed Oct 24 12:10:01 2007
From: chandrashekhar.mullaparthi@REDACTED (Chandru)
Date: Wed, 24 Oct 2007 11:10:01 +0100
Subject: [erlang-questions] http:request with delete method
In-Reply-To: <13361032.post@talk.nabble.com>
References: <13361032.post@talk.nabble.com>
Message-ID: <b447a30d0710240310r55ad82d7se57f3a52f632ca1d@mail.gmail.com>

On 23/10/2007, Zvi <exta7@REDACTED> wrote:
>
> Anyone know, how to perform HTTP DELETE method with inets?
>
> Is  this is a right way:
>
>  {ok, Response} = http:request(delete, {URL,[]}, [], [{sync,true}]).
>
> Also is there ibrowse with driver compiled for Windows? I tried to install
> it via CEAN, but it installed without driver.
>

Zvi, the driver is there for historical reasons and is not used by
ibrowse. So you can use it on Windows.

cheers
Chandru


From bengt.kleberg@REDACTED  Wed Oct 24 12:22:20 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 24 Oct 2007 12:22:20 +0200
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
References: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
Message-ID: <471F1CDC.2090909@ericsson.com>

greetings,

the first 3 solutions will be gone after the process dies. ets tables 
are garbage collected when their ''owning process'' dies.

i would recommend 1 until you get > 1000 items per process.
or you could start with 1. keep the adding/removing/extracting hidden in 
a module and then switch to 3 using another module with the same name if 
you run into problems.


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-24 12:00, Alexander Lamb wrote:
> Hello again,
> 
> So, I am gradually understanding the design philosophy of an Erlang 
> application.
> On the one hand, functionnal programming is about having no state. 
> However, in the situation of a Web application, you obviously need to 
> keep some state otherwise you are back to PhP style (in the sense you 
> are recalculating everything at each request).
> 
> I went back to Joe's book and read again the chapter on processes. And, 
> as someone suggested earlier, I realized the state of an Erlang program 
> can actually be kept in a process. So when designing an application, you 
> indeed model the processes, then the state of the processes, then the 
> messages.
> 
> So for example, if I create a process for each user who logs in to a 
> program. Upon creation of the process I could fetch all the features and 
> worklists the user has, then keep them in his process until he logs out 
> (or after a given long timeout).
> 
> I can do that for every task which needs to keep a state between requests.
> 
> Therefore, I have a question: where is it usuall or reasonnable to keep 
> the state of a process?
> 
> 1) in the parameters of the function we spawn
> 2) in the process dictionnary
> 3) in an ETS structure
> 4) in a in-memory mnesia structure
> 
> I suppose it really depends on the amount of data and how it will be 
> modified (and if we want to keep persistent some session data after the 
> process is killed).
> 
> Tell me if I am wrong, but the first two solutions will put data on the 
> process stack and will be destroyed when the process is killed. The two 
> other solutions will have to be cleaned out when the process is killed.
> 
> Are there some guidelines?
> 
> For a user process, I might have 30 or 40 features in a list, some 
> values about his account, etc...
> For a process managing (for example) browsing patients in a list, I will 
> have the list of patients with the visits for the currently selected 
> patient, etc...
> 
> Thanks for any hints!
> 
> Alex
> --
> Alexander Lamb
> Founding Associate
> RODANOTECH S?rl
> 
> 4 ch. de la Tour de Champel
> 1206 Geneva
> Switzerland
> 
> Tel:  022 347 77 37
> Fax: 022 347 77 38
> 
> http://www.rodanotech.ch
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From thomasl_erlang@REDACTED  Wed Oct 24 11:41:15 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Wed, 24 Oct 2007 02:41:15 -0700 (PDT)
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
Message-ID: <417604.65860.qm@web38806.mail.mud.yahoo.com>



--- Steve Vinoski <vinoski@REDACTED> wrote:

 Anders, thanks for collecting and posting these.
> I've just performed a set
> of new timings for all of them, as listed below. For
> each, I just ran this
> command:
> 
> time erl -smp -noshell -run <test_case> main
> o1000k.ap >/dev/null
> 
> where "<test_case>" is the name of the tbray test
> case file. All were
> looped ten times, and I took the best timing for
> each. All tests were done
> on my 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM
> Linux box, in a local
> (non-NFS) directory.

So, looking at Steve's results on his 8-core system,
we have:

            real    user     tbray5/real    user/real
tbray5      9.8     --       1.0            --
tbray14     6.63    34.53    1.48           5.21
tbray15     4.12    25.14    2.38           6.10
tbray16     3.16    16.15    3.10           5.11
tbray_tuple 2.28     8.61    4.30           3.78
tbray_ets   1.87     7.42    5.24           3.97
tbray_blkr  1.55     7.34    6.32           4.74

tbray5/real is the speedup versus the baseline, while
user/real is the speedup for each version due to
parallelization.

Thus, the latest version is 6.3 times faster than the
first one. The parallel speedup is about the same in
tbray5 and tbray_blkr, a very decent utilization of
>50%, but the amount of work (user) has shrunk from
(presumably more than) 34.53 seconds to 7.34 seconds.

Tim Bray's original Erlang number on "his macbook"
appears to be 34.16 seconds user (probably about the
same real?). How does this compare to Ruby? Tim Bray
reported that it needed 3.46 seconds real, again on
his macbook. (As I understand it, all results here are
for the big data set.)

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From me@REDACTED  Wed Oct 24 13:05:15 2007
From: me@REDACTED (KatolaZ)
Date: Wed, 24 Oct 2007 13:05:15 +0200
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
References: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
Message-ID: <20071024110514.GI3144@katolaz.homeunix.net>

On Wed, Oct 24, 2007 at 12:00:31PM +0200, Alexander Lamb wrote:
>
> Therefore, I have a question: where is it usuall or reasonnable to keep the 
> state of a process?
>
> 1) in the parameters of the function we spawn
> 2) in the process dictionnary
> 3) in an ETS structure
> 4) in a in-memory mnesia structure

All those solutions imply to have the process on the server side 
running for the whole duration of a session. If the process crashes, 
you loose the state, and all other stuff. 

This is not the exact replacement of what happens with sessions in
PHP, where the state (session variables) are usually stored into a
file or into a DB. 

Let's put it this way: a web application is one of the few exceptions
to the usually-working golden rule: "always avoid side effects and
state-related operations that can survive a process death".

This is basically why I sometimes use dets tables for web-based
erlang applications. They're somehow fast and performant and exactly 
surrogate PHP or Python sessions. 

HND

Enzo

-- 
[ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ]
[ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ]
[ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ]
[ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ]


From vinoski@REDACTED  Wed Oct 24 14:00:55 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Wed, 24 Oct 2007 08:00:55 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <417604.65860.qm@web38806.mail.mud.yahoo.com>
References: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
	<417604.65860.qm@web38806.mail.mud.yahoo.com>
Message-ID: <65b2728e0710240500k7a178721pf53aa591f766a633@mail.gmail.com>

On 10/24/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> Anders, thanks for collecting and posting these.
> > I've just performed a set
> > of new timings for all of them, as listed below. For
> > each, I just ran this
> > command:
> >
> > time erl -smp -noshell -run <test_case> main
> > o1000k.ap >/dev/null
> >
> > where "<test_case>" is the name of the tbray test
> > case file. All were
> > looped ten times, and I took the best timing for
> > each. All tests were done
> > on my 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM
> > Linux box, in a local
> > (non-NFS) directory.
>
> So, looking at Steve's results on his 8-core system,
> we have:
>
>             real    user     tbray5/real    user/real
> tbray5      9.8     --       1.0            --
> tbray14     6.63    34.53    1.48           5.21
> tbray15     4.12    25.14    2.38           6.10
> tbray16     3.16    16.15    3.10           5.11
> tbray_tuple 2.28     8.61    4.30           3.78
> tbray_ets   1.87     7.42    5.24           3.97
> tbray_blkr  1.55     7.34    6.32           4.74
>
> tbray5/real is the speedup versus the baseline, while
> user/real is the speedup for each version due to
> parallelization.
>
> Thus, the latest version is 6.3 times faster than the
> first one. The parallel speedup is about the same in
> tbray5 and tbray_blkr, a very decent utilization of
> >50%, but the amount of work (user) has shrunk from
> (presumably more than) 34.53 seconds to 7.34 seconds.
>
> Tim Bray's original Erlang number on "his macbook"
> appears to be 34.16 seconds user (probably about the
> same real?). How does this compare to Ruby? Tim Bray
> reported that it needed 3.46 seconds real, again on
> his macbook. (As I understand it, all results here are
> for the big data set.)
>

Yes, all results are for o1000k.ap, Tim's original large dataset. As for
Ruby, I just ran Tim's original code on the 8-core, and out of ten attempts
the best was:

real    0m2.210s
user    0m2.095s
sys     0m0.109s

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/0ab0ca83/attachment.htm>

From alexander.lamb@REDACTED  Wed Oct 24 14:15:13 2007
From: alexander.lamb@REDACTED (Alexander Lamb)
Date: Wed, 24 Oct 2007 14:15:13 +0200
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <471F1CDC.2090909@ericsson.com>
References: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
	<471F1CDC.2090909@ericsson.com>
Message-ID: <8F51C3C4-87E7-4AB5-8F72-1CCA7E8B79F9@rodanotech.ch>

Thanks a lot,

But you seem to not consider solution 2 (process dictionnary). Is  
this because it is not efficient, or at least not more efficient than  
parameters?

Also, I read somewhere that the heap for processes in Erlang was  
rather small for historical reasons. If I store data in the  
parameters or in the process dictionnary, will I run into some limit?
Alternatively, I suppose the data for ETS structures are stored on  
the node's heap and not in the process heap? Is this correct?

Thanks,

Alex

Le 24 oct. 07 ? 12:22, Bengt Kleberg a ?crit :

> greetings,
>
> the first 3 solutions will be gone after the process dies. ets tables
> are garbage collected when their ''owning process'' dies.
>
> i would recommend 1 until you get > 1000 items per process.
> or you could start with 1. keep the adding/removing/extracting  
> hidden in
> a module and then switch to 3 using another module with the same  
> name if
> you run into problems.
>
>
> bengt
> Those were the days...
>     EPO guidelines 1978: "If the contribution to the known art resides
>     solely in a computer program then the subject matter is not
>     patentable in whatever manner it may be presented in the claims."
>
>
> On 2007-10-24 12:00, Alexander Lamb wrote:
>> Hello again,
>>
>> So, I am gradually understanding the design philosophy of an Erlang
>> application.
>> On the one hand, functionnal programming is about having no state.
>> However, in the situation of a Web application, you obviously need to
>> keep some state otherwise you are back to PhP style (in the sense you
>> are recalculating everything at each request).
>>
>> I went back to Joe's book and read again the chapter on processes.  
>> And,
>> as someone suggested earlier, I realized the state of an Erlang  
>> program
>> can actually be kept in a process. So when designing an  
>> application, you
>> indeed model the processes, then the state of the processes, then the
>> messages.
>>
>> So for example, if I create a process for each user who logs in to a
>> program. Upon creation of the process I could fetch all the  
>> features and
>> worklists the user has, then keep them in his process until he  
>> logs out
>> (or after a given long timeout).
>>
>> I can do that for every task which needs to keep a state between  
>> requests.
>>
>> Therefore, I have a question: where is it usuall or reasonnable to  
>> keep
>> the state of a process?
>>
>> 1) in the parameters of the function we spawn
>> 2) in the process dictionnary
>> 3) in an ETS structure
>> 4) in a in-memory mnesia structure
>>
>> I suppose it really depends on the amount of data and how it will be
>> modified (and if we want to keep persistent some session data  
>> after the
>> process is killed).
>>
>> Tell me if I am wrong, but the first two solutions will put data  
>> on the
>> process stack and will be destroyed when the process is killed.  
>> The two
>> other solutions will have to be cleaned out when the process is  
>> killed.
>>
>> Are there some guidelines?
>>
>> For a user process, I might have 30 or 40 features in a list, some
>> values about his account, etc...
>> For a process managing (for example) browsing patients in a list,  
>> I will
>> have the list of patients with the visits for the currently selected
>> patient, etc...
>>
>> Thanks for any hints!
>>
>> Alex
>> --
>> Alexander Lamb
>> Founding Associate
>> RODANOTECH S?rl
>>
>> 4 ch. de la Tour de Champel
>> 1206 Geneva
>> Switzerland
>>
>> Tel:  022 347 77 37
>> Fax: 022 347 77 38
>>
>> http://www.rodanotech.ch
>>
>>
>>
>>
>> --------------------------------------------------------------------- 
>> ---
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

--
Alexander Lamb
Founding Associate
RODANOTECH S?rl

4 ch. de la Tour de Champel
1206 Geneva
Switzerland

Tel:  022 347 77 37
Fax: 022 347 77 38

http://www.rodanotech.ch



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/f32e6e49/attachment.htm>

From bengt.kleberg@REDACTED  Wed Oct 24 15:21:27 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 24 Oct 2007 15:21:27 +0200
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <8F51C3C4-87E7-4AB5-8F72-1CCA7E8B79F9@rodanotech.ch>
References: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
	<471F1CDC.2090909@ericsson.com>
	<8F51C3C4-87E7-4AB5-8F72-1CCA7E8B79F9@rodanotech.ch>
Message-ID: <471F46D7.70903@ericsson.com>

greetings,

i did not consider using 2 (process dictionary) since i have found it to 
be a problem when writing tests and also when trying to figure out what 
went wrong on a live system.

the heap for a process will grow when needed. and shrink, too. so you 
should not worry about it starting out small. unless you know that it 
must become big and want to make things a small amount faster.

since ets is garbage collected with the process i would assume (i could 
be wrong) that ets uses the process heap.


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-24 14:15, Alexander Lamb wrote:
> Thanks a lot,
> 
> But you seem to not consider solution 2 (process dictionnary). Is this 
> because it is not efficient, or at least not more efficient than parameters?
> 
> Also, I read somewhere that the heap for processes in Erlang was rather 
> small for historical reasons. If I store data in the parameters or in 
> the process dictionnary, will I run into some limit?
> Alternatively, I suppose the data for ETS structures are stored on the 
> node's heap and not in the process heap? Is this correct?
> 
> Thanks,
> 
> Alex
> 
> Le 24 oct. 07 ? 12:22, Bengt Kleberg a ?crit :
> 
>> greetings,
>>
>> the first 3 solutions will be gone after the process dies. ets tables 
>> are garbage collected when their ''owning process'' dies.
>>
>> i would recommend 1 until you get > 1000 items per process.
>> or you could start with 1. keep the adding/removing/extracting hidden in 
>> a module and then switch to 3 using another module with the same name if 
>> you run into problems.
>>
>>
>> bengt
>> Those were the days...
>>     EPO guidelines 1978: "If the contribution to the known art resides
>>     solely in a computer program then the subject matter is not
>>     patentable in whatever manner it may be presented in the claims."
>>
>>
>> On 2007-10-24 12:00, Alexander Lamb wrote:
>>> Hello again,
>>>
>>> So, I am gradually understanding the design philosophy of an Erlang 
>>> application.
>>> On the one hand, functionnal programming is about having no state. 
>>> However, in the situation of a Web application, you obviously need to 
>>> keep some state otherwise you are back to PhP style (in the sense you 
>>> are recalculating everything at each request).
>>>
>>> I went back to Joe's book and read again the chapter on processes. And, 
>>> as someone suggested earlier, I realized the state of an Erlang program 
>>> can actually be kept in a process. So when designing an application, you 
>>> indeed model the processes, then the state of the processes, then the 
>>> messages.
>>>
>>> So for example, if I create a process for each user who logs in to a 
>>> program. Upon creation of the process I could fetch all the features and 
>>> worklists the user has, then keep them in his process until he logs out 
>>> (or after a given long timeout).
>>>
>>> I can do that for every task which needs to keep a state between 
>>> requests.
>>>
>>> Therefore, I have a question: where is it usuall or reasonnable to keep 
>>> the state of a process?
>>>
>>> 1) in the parameters of the function we spawn
>>> 2) in the process dictionnary
>>> 3) in an ETS structure
>>> 4) in a in-memory mnesia structure
>>>
>>> I suppose it really depends on the amount of data and how it will be 
>>> modified (and if we want to keep persistent some session data after the 
>>> process is killed).
>>>
>>> Tell me if I am wrong, but the first two solutions will put data on the 
>>> process stack and will be destroyed when the process is killed. The two 
>>> other solutions will have to be cleaned out when the process is killed.
>>>
>>> Are there some guidelines?
>>>
>>> For a user process, I might have 30 or 40 features in a list, some 
>>> values about his account, etc...
>>> For a process managing (for example) browsing patients in a list, I will 
>>> have the list of patients with the visits for the currently selected 
>>> patient, etc...
>>>
>>> Thanks for any hints!
>>>
>>> Alex
>>> --
>>> Alexander Lamb
>>> Founding Associate
>>> RODANOTECH S?rl
>>>
>>> 4 ch. de la Tour de Champel
>>> 1206 Geneva
>>> Switzerland
>>>
>>> Tel:  022 347 77 37
>>> Fax: 022 347 77 38
>>>
>>> http://www.rodanotech.ch
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> erlang-questions mailing list
>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> --
> Alexander Lamb
> Founding Associate
> RODANOTECH S?rl
> 
> 4 ch. de la Tour de Champel
> 1206 Geneva
> Switzerland
> 
> Tel:  022 347 77 37
> Fax: 022 347 77 38
> 
> http://www.rodanotech.ch
> 
> 
> 


From bbmaj7@REDACTED  Wed Oct 24 15:27:54 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Wed, 24 Oct 2007 23:27:54 +1000 (EST)
Subject: [erlang-questions] Efficiency of splitting binaries
Message-ID: <68680.53297.qm@web52012.mail.re2.yahoo.com>

I'm looking for advice on efficiently splitting a binary into smaller chunks.

I currently use iolist_size() to check the size of the input binary then
recursively break off a chunk from the front with something like, eg.

    << ChunkData:30/binary-unit:8, RestData/binary >> = Data

... then use RestData as input to the next cycle.

Is this sensible? Are there more efficient approaches?
What effect does the binary-unit specifier have on performance? eg.

    ChunkData:1/binary-unit:240
versus
    ChunkData:240/binary



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From jao@REDACTED  Wed Oct 24 15:48:09 2007
From: jao@REDACTED (Jack Orenstein)
Date: Wed, 24 Oct 2007 09:48:09 -0400
Subject: [erlang-questions] Forming a cluster
Message-ID: <758693AE-231D-45F7-A9AD-A312754FB1B9@geophile.com>

What is the recommended procedure for setting up a cluster of nodes,  
each running an Erlang VM (all with the same .erlang.cookie)? The  
nodes are all in the same subnet, behind a firewall. The node  
population is reasonably static, but nodes there may be occasional  
additions. When a node joins, the nodes already in the cluster need  
to know about it. Each node has to know about every other node, (e.g.  
by calling nodes()).

net_kernel:monitor_nodes will indicate when a node joins, but only if  
the new node sends a message to some node in the cluster. How does  
the new node know who is present? There seems to be a bootstrapping  
problem. Also, requiring a new node to know about a cluster member is  
problematic since the selected node might be down.

net_adm and the .erlang.hosts file is a possible solution, but that  
seems static. It doesn't seem to accomodate nodes joining after the  
initial cluster forms.

Is there a dynamic solution that doesn't require the new node to know  
anything about the existing cluster members?

Jack Orenstein


From per.gustafsson@REDACTED  Wed Oct 24 15:55:21 2007
From: per.gustafsson@REDACTED (Per Gustafsson)
Date: Wed, 24 Oct 2007 15:55:21 +0200
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <68680.53297.qm@web52012.mail.re2.yahoo.com>
References: <68680.53297.qm@web52012.mail.re2.yahoo.com>
Message-ID: <471F4EC9.5030706@it.uu.se>

Richard Andrews wrote:
> I'm looking for advice on efficiently splitting a binary into smaller chunks.
> 
> I currently use iolist_size() to check the size of the input binary then
> recursively break off a chunk from the front with something like, eg.
> 

Why are you using iolist_size()? Is it possiblethat your input is not a 
binary?


>     << ChunkData:30/binary-unit:8, RestData/binary >> = Data
> 

This should be efficient you are creating sub-binaries for both 
ChunkData and RestData so the cost should be independent of their 
respective sizes.

> ... then use RestData as input to the next cycle.
> 
> Is this sensible? Are there more efficient approaches?
> What effect does the binary-unit specifier have on performance? eg.
> 
>     ChunkData:1/binary-unit:240
> versus
>     ChunkData:240/binary
> 
> 

As long as Size*Unit remains the same this should not affect the runtime 
at all, but note that ChunkData:240/binary really means 
ChunkData:240/binary-unit:8

Per Gustafsson


From bjorn@REDACTED  Wed Oct 24 16:06:33 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 24 Oct 2007 16:06:33 +0200
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <68680.53297.qm@web52012.mail.re2.yahoo.com>
References: <68680.53297.qm@web52012.mail.re2.yahoo.com>
Message-ID: <o11wbkhaja.fsf@erix.ericsson.se>

Richard Andrews <bbmaj7@REDACTED> writes:

> I'm looking for advice on efficiently splitting a binary into smaller chunks.
> 
> I currently use iolist_size() to check the size of the input binary then
> recursively break off a chunk from the front with something like, eg.
> 
>     << ChunkData:30/binary-unit:8, RestData/binary >> = Data

unit:8 is default for binaries, so you don't have to specify it.

> ... then use RestData as input to the next cycle.
> 
> Is this sensible? Are there more efficient approaches?
> What effect does the binary-unit specifier have on performance? eg.
> 
>     ChunkData:1/binary-unit:240
> versus
>     ChunkData:240/binary

None if both the size and unit are known at compile-time.

If the size is variable, the size and unit values will be multiplied at run-time.
The run-time system may optimize common sizes such as 1 and 8.

I recommend that you don't change the unit size for binaries, but use the
default 8.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From thomasl_erlang@REDACTED  Wed Oct 24 16:56:46 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Wed, 24 Oct 2007 07:56:46 -0700 (PDT)
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <8F51C3C4-87E7-4AB5-8F72-1CCA7E8B79F9@rodanotech.ch>
Message-ID: <841547.66275.qm@web38804.mail.mud.yahoo.com>


--- Alexander Lamb <alexander.lamb@REDACTED>
wrote:

> But you seem to not consider solution 2 (process
> dictionnary). Is  
> this because it is not efficient, or at least not
> more efficient than  
> parameters?

I'd recommend not relying on the process dictionary.
Granted, there are cases when it is useful, but you
get all the problems of (process-)global variables
too.

> Also, I read somewhere that the heap for processes
> in Erlang was  
> rather small for historical reasons. If I store data
> in the  
> parameters or in the process dictionnary, will I run
> into some limit?

A process heap is by default small _initially_, but it
will grow dynamically on demand. It's just an
optimization to permit plenty of processes without
running out of memory.

> Alternatively, I suppose the data for ETS structures
> are stored on  
> the node's heap and not in the process heap? Is this
> correct?

As far as I know, yes. However, reading data from ets
copies it into the process heap. (As I understand it,
this is true even for the hybrid runtime system.)

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From xapwing@REDACTED  Wed Oct 24 16:59:31 2007
From: xapwing@REDACTED (Arie van Wingerden)
Date: Wed, 24 Oct 2007 16:59:31 +0200
Subject: [erlang-questions] ctrl-g disabled in erl
Message-ID: <5949aaa30710240759g7911d11fhd81508ea72c1e8fe@mail.gmail.com>

Hi,

when I start ERL in Windows I see:
   Eshell V5.5.5  (abort with ^G)
   1>
but using ctrl+g won't bring up the jcl prompt (as suggested in the book
"Programming Erlang" on page 21).
I read some postings on the web discussing the dangers of ctrl+g

Questions:
   1. is the disabling of ctrl+g (to invoke jcl prompt) intentional?
   2. can it be enabled?

BTW: it does work in WERL

TIA,
   Arie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/af82ddce/attachment.htm>

From rpettit@REDACTED  Wed Oct 24 17:12:40 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Wed, 24 Oct 2007 10:12:40 -0500
Subject: [erlang-questions] Forming a cluster
In-Reply-To: <758693AE-231D-45F7-A9AD-A312754FB1B9@geophile.com>
References: <758693AE-231D-45F7-A9AD-A312754FB1B9@geophile.com>
Message-ID: <20071024151240.GC26789@vailsys.com>

On Wed, Oct 24, 2007 at 09:48:09AM -0400, Jack Orenstein wrote:
> What is the recommended procedure for setting up a cluster of nodes,  
> each running an Erlang VM (all with the same .erlang.cookie)? The  
> nodes are all in the same subnet, behind a firewall. The node  
> population is reasonably static, but nodes there may be occasional  
> additions. When a node joins, the nodes already in the cluster need  
> to know about it. Each node has to know about every other node, (e.g.  
> by calling nodes()).
> 
> net_kernel:monitor_nodes will indicate when a node joins, but only if  
> the new node sends a message to some node in the cluster. How does  
> the new node know who is present? There seems to be a bootstrapping  
> problem. Also, requiring a new node to know about a cluster member is  
> problematic since the selected node might be down.
> 
> net_adm and the .erlang.hosts file is a possible solution, but that  
> seems static. It doesn't seem to accomodate nodes joining after the  
> initial cluster forms.
> 
> Is there a dynamic solution that doesn't require the new node to know  
> anything about the existing cluster members?

Most systems I've worked on involved a shared set of mnesia database nodes
containing configuration information for the entire cluster.

All other nodes in the cluster are started with -mnesia extra_db_nodes [...],
and thus no explicit net_adm:ping/1 is required (in my code).

Of course this is just another form of static configuration...

To do what you want you may need to take a close look at how epmd(1) works.

-Rick


From xapwing@REDACTED  Wed Oct 24 17:38:55 2007
From: xapwing@REDACTED (Arie van Wingerden)
Date: Wed, 24 Oct 2007 17:38:55 +0200
Subject: [erlang-questions] noob question on CEAN
Message-ID: <5949aaa30710240838o2686e64bjdb5d62bd79b75dd2@mail.gmail.com>

Hi,

i have played some time using the native installation of Erlang from
www.erlang.org, which worked just fine.

In order to get aquainted with CEAN i uninstalled Erlang and installed CEAN
instead, unpacking the archive under "c:\program files" in WinXP.
When I start the Erlang shell using start.bat in "c:\program files\cean" the
shell starts fine.
When I try to evaluate some cean function I always get an error.
Like so:

Eshell V5.5.5  (abort with ^G)
1> cean:help().

=ERROR REPORT==== 24-Oct-2007::17:32:58 ===
Error in process <0.30.0> with exit value:
{undef,[{cean,help,[]},{erl_eval,do_a
pply,5},{shell,exprs,6},{shell,eval_loop,3}]}

** exited: {undef,[{cean,help,[]},
                   {erl_eval,do_apply,5},
                   {shell,exprs,6},
                   {shell,eval_loop,3}]} **
2>

What obvious point I am missing here?

TIA,
   Arie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/2aadf5a0/attachment.htm>

From christophe.romain@REDACTED  Wed Oct 24 17:53:52 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Wed, 24 Oct 2007 17:53:52 +0200
Subject: [erlang-questions] noob question on CEAN
In-Reply-To: <5949aaa30710240838o2686e64bjdb5d62bd79b75dd2@mail.gmail.com>
References: <5949aaa30710240838o2686e64bjdb5d62bd79b75dd2@mail.gmail.com>
Message-ID: <AA78F68D-E66D-4E46-BBC3-284ADA1B19DC@process-one.net>

may that be related to the space from "program files" ?
do you get the same result if installing into c:\cean ?


From frank.blecha@REDACTED  Wed Oct 24 17:54:38 2007
From: frank.blecha@REDACTED (Frank Blecha)
Date: Wed, 24 Oct 2007 09:54:38 -0600
Subject: [erlang-questions] erlang api documentation
Message-ID: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>

Is there a hyper linked version out there somewhere?  I have been using the
module docs from erlang.org, but a linked version would be a bit handier.
Thanks.

- Frank

-- 
Frank Blecha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/874dd6b7/attachment.htm>

From frank.blecha@REDACTED  Wed Oct 24 17:54:38 2007
From: frank.blecha@REDACTED (Frank Blecha)
Date: Wed, 24 Oct 2007 09:54:38 -0600
Subject: [erlang-questions] erlang api documentation
Message-ID: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>

Is there a hyper linked version out there somewhere?  I have been using the
module docs from erlang.org, but a linked version would be a bit handier.
Thanks.

- Frank

-- 
Frank Blecha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/874dd6b7/attachment-0001.htm>

From peter@REDACTED  Wed Oct 24 18:18:27 2007
From: peter@REDACTED (Peter K Chan)
Date: Wed, 24 Oct 2007 18:18:27 +0200
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
Message-ID: <A250ACDE71BB1847AB72F6D84DB4F3B533B116@ntxbeus24.exchange.xchg>

Have you tried gotapi.com? It has a very nice search-based interface for
a host of languages, including Erlang.

 

Peter

 

From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Frank Blecha
Sent: Wednesday, October 24, 2007 10:55 AM
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] erlang api documentation

 

Is there a hyper linked version out there somewhere?  I have been using
the module docs from erlang.org, but a linked version would be a bit
handier. Thanks.
 
- Frank

-- 
Frank Blecha

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/f3ecec65/attachment.htm>

From kenneth.lundin@REDACTED  Wed Oct 24 19:03:50 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Wed, 24 Oct 2007 19:03:50 +0200
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
Message-ID: <de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>

Hi Frank,

Please explain what you mean with "hyper linked" version and why you don't as it
seems, think that the available documentation on erlang.org
http://erlang.org/doc/

is hyperlinked?

We are well aware of that the html format is not modern and that it
would be good with better search facilities but I don't know if that
is what you are after.

/Kenneth Erlang/OTP team at Ericsson

On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> Is there a hyper linked version out there somewhere?  I have been using the
> module docs from erlang.org, but a linked version would be a bit handier.
> Thanks.
>
> - Frank
>
> --
> Frank Blecha
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bent@REDACTED  Wed Oct 24 19:36:08 2007
From: bent@REDACTED (Ben Munat)
Date: Wed, 24 Oct 2007 07:36:08 -1000
Subject: [erlang-questions] Erlang processes and state management design
In-Reply-To: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
References: <686CBE68-7213-4EDC-9369-873EADD89F57@rodanotech.ch>
Message-ID: <471F8288.4070108@munat.com>

Alexander Lamb wrote:
> Hello again,
> 
> So, I am gradually understanding the design philosophy of an Erlang 
> application.
> On the one hand, functionnal programming is about having no state. 
> However, in the situation of a Web application, you obviously need to 
> keep some state otherwise you are back to PhP style (in the sense you 
> are recalculating everything at each request).

"Back to PHP style"? That's not PHP style... that is called "shared nothing" and 
is used in web apps in a lot of languages and frameworks. From my experience, 
reconstituting state on each request is highly recommended... it's the norm. I 
always keep session data in the database as well.

You should check out erlyweb... it offers a database layer that simplifies 
access to data in the function handling a request. Not sure how it handles 
sessions... that's probably a feature of yaws though.

Yariv? You lurking out there? Care to promote your framework here?

Ben


> I went back to Joe's book and read again the chapter on processes. And, 
> as someone suggested earlier, I realized the state of an Erlang program 
> can actually be kept in a process. So when designing an application, you 
> indeed model the processes, then the state of the processes, then the 
> messages.
> 
> So for example, if I create a process for each user who logs in to a 
> program. Upon creation of the process I could fetch all the features and 
> worklists the user has, then keep them in his process until he logs out 
> (or after a given long timeout).
> 
> I can do that for every task which needs to keep a state between requests.
> 
> Therefore, I have a question: where is it usuall or reasonnable to keep 
> the state of a process?
> 
> 1) in the parameters of the function we spawn
> 2) in the process dictionnary
> 3) in an ETS structure
> 4) in a in-memory mnesia structure
> 
> I suppose it really depends on the amount of data and how it will be 
> modified (and if we want to keep persistent some session data after the 
> process is killed).
> 
> Tell me if I am wrong, but the first two solutions will put data on the 
> process stack and will be destroyed when the process is killed. The two 
> other solutions will have to be cleaned out when the process is killed.
> 
> Are there some guidelines?
> 
> For a user process, I might have 30 or 40 features in a list, some 
> values about his account, etc...
> For a process managing (for example) browsing patients in a list, I will 
> have the list of patients with the visits for the currently selected 
> patient, etc...
> 
> Thanks for any hints!
> 
> Alex
> --
> Alexander Lamb
> Founding Associate
> RODANOTECH S?rl
> 
> 4 ch. de la Tour de Champel
> 1206 Geneva
> Switzerland
> 
> Tel:  022 347 77 37
> Fax: 022 347 77 38
> 
> http://www.rodanotech.ch
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From fritchie@REDACTED  Wed Oct 24 19:38:01 2007
From: fritchie@REDACTED (Scott Lystig Fritchie)
Date: Wed, 24 Oct 2007 12:38:01 -0500
Subject: [erlang-questions] turning up debugging on ports
In-Reply-To: Message of "Wed, 24 Oct 2007 13:40:47 +1000."
	<471EBEBF.4090304@ghostgun.com> 
Message-ID: <200710241738.l9OHc1H4048852@snookles.snookles.com>

>>>>> "jm" == jm  <jeffm@REDACTED> writes:

jm> Is there an option to erl which will turn up the debugging so I
jm> can see what erlang is doing in relation to a port module I'm
jm> attempting to debug.

One way is to turn on the Erlang tracing facility on the Erlang side
of the world.  See the "dbg" module in the docs (Tool Applications,
then runtime_tools, then dbg).

A small set of functions, originally posted by Serge Aleynikov, were a
big help in starting to use dbg, which is a *very* low level tool.
After seeing what these functions do, the "oh, yeah" moments follow
quickly when you need to do something different.

-Scott

--- snip --- snip --- snip --- snip --- snip --- snip --- snip ---

%% Put this in your .erlang file:

%% Shell hacks from Serge
code:load_abs("/path/to/user_default").

--- snip --- snip --- snip --- snip --- snip --- snip --- snip ---

%% user_default.erl

-module(user_default).
-author('serge@REDACTED').

-export([help/0,dbgtc/1, dbgon/1, dbgon/2,
         dbgadd/1, dbgadd/2, dbgdel/1, dbgdel/2, dbgoff/0,
         l/0]).

-import(io, [format/1]).

help() ->
     shell_default:help(),
     format("** user extended commands **~n"),
     format("dbgtc(File)   -- use dbg:trace_client() to read data from File\n"),
     format("dbgon(M)      -- enable dbg tracer on all funs in module M\n"),
     format("dbgon(M,Fun)  -- enable dbg tracer for module M and function F\n"),
     format("dbgon(M,File) -- enable dbg tracer for module M and log to File\n"),
     format("dbgadd(M)     -- enable call tracer for module M\n"),
     format("dbgadd(M,F)   -- enable call tracer for function M:F\n"),
     format("dbgdel(M)     -- disable call tracer for module M\n"),
     format("dbgdel(M,F)   -- disable call tracer for function M:F\n"),
     format("dbgoff()      -- disable dbg tracer (calls dbg:stop/0)\n"),
     format("l()           -- load all changed modules\n"),
%     format("nl()          -- load all changed modules on all known nodes\n"),
     format("mm()          -- list modified modules\n"),
     true.

dbgtc(File) ->
    Fun = fun({trace,_,call,{M,F,A}}, _) ->
                 io:format("call: ~w:~w~w~n", [M,F,A]);
             ({trace,_,return_from,{M,F,A},R}, _) ->
                 io:format("retn: ~w:~w/~w -> ~w~n", [M,F,A,R]);
             (A,B) ->
                 io:format("~w: ~w~n", [A,B])
          end,
    dbg:trace_client(file, File, {Fun, []}).

dbgon(Module) ->
    case dbg:tracer() of
    {ok,_} ->
       dbg:p(all,call),
       dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
       ok;
    Else ->
       Else
    end.

dbgon(Module, Fun) when is_atom(Fun) ->
    {ok,_} = dbg:tracer(),
    dbg:p(all,call),
    dbg:tpl(Module, Fun, [{'_',[],[{return_trace}]}]),
    ok;

dbgon(Module, File) when is_list(File) ->
    {ok,_} = dbg:tracer(file, dbg:trace_port(file, File)),
    dbg:p(all,call),
    dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
    ok.

dbgadd(Module) ->
    dbg:tpl(Module, [{'_',[],[{return_trace}]}]),
    ok.

dbgadd(Module, Fun) ->
    dbg:tpl(Module, Fun, [{'_',[],[{return_trace}]}]),
    ok.

dbgdel(Module) ->
    dbg:ctpl(Module),
    ok.

dbgdel(Module, Fun) ->
    dbg:ctpl(Module, Fun),
    ok.

dbgoff() ->
    dbg:stop().

l() ->
    FiltZip = lists:filter(
	fun({_Mod, Path}) when is_list(Path) ->
		case string:str(Path, "/kernel-") +
		     string:str(Path, "/stdlib-") of
			0 -> true;
			_ -> false
		end;
	    (_) -> false
	end, code:all_loaded()),
    {Ms, _} = lists:unzip(FiltZip),
    lists:foldl(fun(M, Acc) ->
			case shell_default:l(M) of
				{error, _} -> Acc;
				_          -> [M|Acc]
			end
		end, [], Ms).


From frank.blecha@REDACTED  Wed Oct 24 20:53:08 2007
From: frank.blecha@REDACTED (Frank Blecha)
Date: Wed, 24 Oct 2007 12:53:08 -0600
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
Message-ID: <e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>

Ah, apologies for not being more specific.  I've been a Java programmer for
last 10 years, and so I was asking specificaly about documentation that was
more like Sun produces (or even the Ruby docs).  For example, it's nice to
have a link to all the methods at the top of the class/module instead of
scrolling down etc.

I have been able to find answers to my questions via the api, I just want to
find them faster.

Thanks.

- Frank


On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
>
> Hi Frank,
>
> Please explain what you mean with "hyper linked" version and why you don't
> as it
> seems, think that the available documentation on erlang.org
> http://erlang.org/doc/
>
> is hyperlinked?
>
> We are well aware of that the html format is not modern and that it
> would be good with better search facilities but I don't know if that
> is what you are after.
>
> /Kenneth Erlang/OTP team at Ericsson
>
> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> > Is there a hyper linked version out there somewhere?  I have been using
> the
> > module docs from erlang.org, but a linked version would be a bit
> handier.
> > Thanks.
> >
> > - Frank
> >
> > --
> > Frank Blecha
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>



-- 
Frank Blecha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/eacd7b53/attachment.htm>

From kenneth.lundin@REDACTED  Wed Oct 24 21:41:12 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Wed, 24 Oct 2007 21:41:12 +0200
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
Message-ID: <de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>

Hi again

I have plans to improve the doc's in the direction you ask for.
I am also considering using Javascript in a similar way as used in the
previously
mentioned http://gotapi.com.
But I think it is still a requirement that the doc's can be downloaded
and browsed
locally without the need for having a web-server running. I think it
would be ok to
require a browser with javascript capabilities (we already require
javascript when it
comes to links between applications)

The suggested approach in summary:
- Stop using frames, Use CSS instead for the layout
- Nicer fonts
- Use Javascript to achieve expandable tree view navigation to the left
- Search as you type support (also with Javascript) similar as gotapi.com
...

Any comments on this approach?

/Kenneth Erlang/OTP team at Ericsson

On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> Ah, apologies for not being more specific.  I've been a Java programmer for
> last 10 years, and so I was asking specificaly about documentation that was
> more like Sun produces (or even the Ruby docs).  For example, it's nice to
> have a link to all the methods at the top of the class/module instead of
> scrolling down etc.
>
> I have been able to find answers to my questions via the api, I just want to
> find them faster.
>
> Thanks.
>
> - Frank
>
>
>
> On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
> > Hi Frank,
> >
> > Please explain what you mean with "hyper linked" version and why you don't
> as it
> > seems, think that the available documentation on erlang.org
> > http://erlang.org/doc/
> >
> > is hyperlinked?
> >
> > We are well aware of that the html format is not modern and that it
> > would be good with better search facilities but I don't know if that
> > is what you are after.
> >
> > /Kenneth Erlang/OTP team at Ericsson
> >
> > On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> > > Is there a hyper linked version out there somewhere?  I have been using
> the
> > > module docs from erlang.org, but a linked version would be a bit
> handier.
> > > Thanks.
> > >
> > > - Frank
> > >
> > > --
> > > Frank Blecha
> > >
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> >
>
>
>
> --
> Frank Blecha
>


From frank.blecha@REDACTED  Wed Oct 24 22:17:10 2007
From: frank.blecha@REDACTED (Frank Blecha)
Date: Wed, 24 Oct 2007 14:17:10 -0600
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
Message-ID: <e00abaee0710241317o111cfb98j374e14c1beace83c@mail.gmail.com>

comments inline

On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
>
> Hi again
>
> I have plans to improve the doc's in the direction you ask for.
> I am also considering using Javascript in a similar way as used in the
> previously
> mentioned http://gotapi.com.
> But I think it is still a requirement that the doc's can be downloaded
> and browsed
> locally without the need for having a web-server running.


[fb3] it'd be nicer for travel at any rate.


> I think it
> would be ok to
> require a browser with javascript capabilities (we already require
> javascript when it
> comes to links between applications)
>
> The suggested approach in summary:
> - Stop using frames, Use CSS instead for the layout
> - Nicer fonts
> - Use Javascript to achieve expandable tree view navigation to the left


[fb3] this is the main thing I want.  I don't care if it's on the left (like
Javadocs) or at the top (like ruby docs), I just would like something
similar, if possible.  Right now the docs are very "unix man page" style,
which is great when I'm on unix and I have grep, but a bit less useful when
I'm on windows (which is my primary platform).

I appreciate someone addressing this, and I realize that I haven't offered
to help.  If there's anything I can do, let me know.

- Frank


- Search as you type support (also with Javascript) similar as gotapi.com
> ...
>
> Any comments on this approach?
>
> /Kenneth Erlang/OTP team at Ericsson
>
> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> > Ah, apologies for not being more specific.  I've been a Java programmer
> for
> > last 10 years, and so I was asking specificaly about documentation that
> was
> > more like Sun produces (or even the Ruby docs).  For example, it's nice
> to
> > have a link to all the methods at the top of the class/module instead of
> > scrolling down etc.
> >
> > I have been able to find answers to my questions via the api, I just
> want to
> > find them faster.
> >
> > Thanks.
> >
> > - Frank
> >
> >
> >
> > On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
> > > Hi Frank,
> > >
> > > Please explain what you mean with "hyper linked" version and why you
> don't
> > as it
> > > seems, think that the available documentation on erlang.org
> > > http://erlang.org/doc/
> > >
> > > is hyperlinked?
> > >
> > > We are well aware of that the html format is not modern and that it
> > > would be good with better search facilities but I don't know if that
> > > is what you are after.
> > >
> > > /Kenneth Erlang/OTP team at Ericsson
> > >
> > > On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
> > > > Is there a hyper linked version out there somewhere?  I have been
> using
> > the
> > > > module docs from erlang.org, but a linked version would be a bit
> > handier.
> > > > Thanks.
> > > >
> > > > - Frank
> > > >
> > > > --
> > > > Frank Blecha
> > > >
> > > > _______________________________________________
> > > > erlang-questions mailing list
> > > > erlang-questions@REDACTED
> > > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > > >
> > >
> >
> >
> >
> > --
> > Frank Blecha
> >
>



-- 
Frank Blecha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/cbe8b2c0/attachment.htm>

From per@REDACTED  Wed Oct 24 22:22:28 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 24 Oct 2007 22:22:28 +0200 (CEST)
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <471EBEBA.6030207@ghostgun.com>
Message-ID: <200710242022.l9OKMSr4015930@pluto.hedeland.org>

jm <jeffm@REDACTED> wrote:
>
>jm wrote:
>> Per Hedeland wrote:
>>> jm <jeffm@REDACTED> wrote:
>>>> it appears somthing is causing erlang to close stdin to the port,
>>> Yes, that was pretty clear from your earlier results.
>> 
>> Just had to prove it.
>
>In continuing to debug this port program I recompile with debug enabled 
>(-g) and attached gdb to the running process which resulted in it 
>writting to a broken pipe as shown

Yes, that is also the expected result if it tries to write to the pipe
after Erlang has closed its end. I don't think you actually answered my
question earlier:

>>> Is the port owner process still running? The port gets automatically
>>> closed when the owner exits.

I.e. I'm referring to the Erlang process that opened the port (unless
you have changed the port owner with e.g. port_connect/2). That process
exiting is the most likely cause of the port getting closed from the
Erlang side. Possibly the port could also be closed by the runtime if
you have specifed some more "advanced" "protocol" than
stream/packet/line and the incoming data doesn't follow it (any random
data can be interpreted as being either of those three), but in that
case the port owner should get an exit signal.

--Per


From hokan.stenholm@REDACTED  Wed Oct 24 21:59:49 2007
From: hokan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=)
Date: Wed, 24 Oct 2007 21:59:49 +0200
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
Message-ID: <471FA435.20603@bredband.net>

Kenneth Lundin wrote:
> Hi again
>
> I have plans to improve the doc's in the direction you ask for.
> I am also considering using Javascript in a similar way as used in the
> previously
> mentioned http://gotapi.com.
> But I think it is still a requirement that the doc's can be downloaded
> and browsed
> locally without the need for having a web-server running. 
Being able to use the docs when the network is down is kind of useful ...

You might also want to consider keeping the docs print friendly,
so that one can easily print various chapters and module documents.
> I think it
> would be ok to
> require a browser with javascript capabilities (we already require
> javascript when it
> comes to links between applications)
>
> The suggested approach in summary:
> - Stop using frames, Use CSS instead for the layout
> - Nicer fonts
> - Use Javascript to achieve expandable tree view navigation to the left
> - Search as you type support (also with Javascript) similar as gotapi.com
> ...
>
>   
Sounds reasonable and useful.

> Any comments on this approach?
>
> /Kenneth Erlang/OTP team at Ericsson
>
> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
>   
>> Ah, apologies for not being more specific.  I've been a Java programmer for
>> last 10 years, and so I was asking specificaly about documentation that was
>> more like Sun produces (or even the Ruby docs).  For example, it's nice to
>> have a link to all the methods at the top of the class/module instead of
>> scrolling down etc.
>>
>> I have been able to find answers to my questions via the api, I just want to
>> find them faster.
>>
>> Thanks.
>>
>> - Frank
>>
>>
>>
>> On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
>>     
>>> Hi Frank,
>>>
>>> Please explain what you mean with "hyper linked" version and why you don't
>>>       
>> as it
>>     
>>> seems, think that the available documentation on erlang.org
>>> http://erlang.org/doc/
>>>
>>> is hyperlinked?
>>>
>>> We are well aware of that the html format is not modern and that it
>>> would be good with better search facilities but I don't know if that
>>> is what you are after.
>>>
>>> /Kenneth Erlang/OTP team at Ericsson
>>>
>>> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
>>>       
>>>> Is there a hyper linked version out there somewhere?  I have been using
>>>>         
>> the
>>     
>>>> module docs from erlang.org, but a linked version would be a bit
>>>>         
>> handier.
>>     
>>>> Thanks.
>>>>
>>>> - Frank
>>>>
>>>> --
>>>> Frank Blecha
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>>         
>>
>> --
>> Frank Blecha
>>
>>     
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>   


From g.olgeni@REDACTED  Wed Oct 24 22:08:20 2007
From: g.olgeni@REDACTED (Giacomo Olgeni)
Date: Wed, 24 Oct 2007 22:08:20 +0200 (CEST)
Subject: [erlang-questions] Bidirectional GIOP
Message-ID: <20071024215715.U1567@olgeni.olgeni>

Hello,

I was trying to activate bidirectional GIOP in Orber and found a
ORB_ENV_USE_BI_DIR_IIOP flag in orber_iiop.hrl, however it is not
documented in the Orber guide. Is it considered safe for production
use?

Kind regards,
   Jimmy


From kenneth.lundin@REDACTED  Wed Oct 24 23:40:55 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Wed, 24 Oct 2007 23:40:55 +0200
Subject: [erlang-questions] Forming a cluster
In-Reply-To: <758693AE-231D-45F7-A9AD-A312754FB1B9@geophile.com>
References: <758693AE-231D-45F7-A9AD-A312754FB1B9@geophile.com>
Message-ID: <de422a40710241440o3b7da58cjeadabbdf13deadfb@mail.gmail.com>

By using
net_adm:names(Host)
or
net_adm:names()

The newly started node can find out about other nodes already started.
Of course this requires knowledge of the possible Hosts which can
have running nodes.

You can also read about global and how it can maintain a fully connected
network.
http://erlang.org/doc/man/global.html

/Kenneth Erlang/OTP team at Ericsson

On 10/24/07, Jack Orenstein <jao@REDACTED> wrote:
> What is the recommended procedure for setting up a cluster of nodes,
> each running an Erlang VM (all with the same .erlang.cookie)? The
> nodes are all in the same subnet, behind a firewall. The node
> population is reasonably static, but nodes there may be occasional
> additions. When a node joins, the nodes already in the cluster need
> to know about it. Each node has to know about every other node, (e.g.
> by calling nodes()).
>
> net_kernel:monitor_nodes will indicate when a node joins, but only if
> the new node sends a message to some node in the cluster. How does
> the new node know who is present? There seems to be a bootstrapping
> problem. Also, requiring a new node to know about a cluster member is
> problematic since the selected node might be down.
>
> net_adm and the .erlang.hosts file is a possible solution, but that
> seems static. It doesn't seem to accomodate nodes joining after the
> initial cluster forms.
>
> Is there a dynamic solution that doesn't require the new node to know
> anything about the existing cluster members?
>
> Jack Orenstein
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ok@REDACTED  Wed Oct 24 23:57:58 2007
From: ok@REDACTED (Richard A. O'Keefe)
Date: Thu, 25 Oct 2007 10:57:58 +1300
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <ffl5ki$tmp$1@ger.gmane.org>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>	<de422a40710230457s5e90a20s9f0dfe82228f42ec@mail.gmail.com>	<00A3DB47-BE71-4A49-A6FB-E962434349C9@gmail.com>
	<Pine.LNX.4.64.0710231618170.18789@amrod.du.uab.ericsson.se>
	<ffl5ki$tmp$1@ger.gmane.org>
Message-ID: <DA5BC3F9-7849-4A55-8515-D76D548054BE@cs.otago.ac.nz>

>> On Tue, 23 Oct 2007, Joel Reymont wrote:
>>
>> JR> Take a look at the following [1] and try to visualize an
>> JR> implementation in Erlang. More thoughts after the example.
>> JR>
>> JR> The data:
>> JR>
>> JR> <Export>
>> JR>    <Product>
>> JR>      <SKU>403276</SKU>
>> JR>      <ItemName>Trivet</ItemName>
>> JR>      <CollectionNo>0</CollectionNo>
>> JR>      <Pages>0</Pages>
>> JR>    </Product>
>> JR> </Export>

The first thing I note here is that this is not what I would call
well designed XML.  A good rule of design in general is
"if data are naturally unordered, do not impose any more order on them
than you can help".  Here we have four string properties identified by
name.  XML has a mechanism that is specifically designed for that job:
attributes.  The design rule has an exception "it is sometimes OK to
represent a set or bag as a sequence, provided you ensure that the
results are invariant under permutation."  So in *good* XML this
example is
   <Export>
     <Product SKU="403276" Item="Trivet" Collection="0" Pages="0"/>
   </Export>
There are two more benefits here.  One is that long-winded element
names like ItemName and CollectionNo can be replaced by shorter
attribute names like Item and Collection because attributes are
inherently contextual: it's really Product@REDACTED and Product@REDACTED
The second is that the attribute version is 83 bytes,
while the element version is 144 bytes, both using 1 space per level
indentation.  That reduces space by >1.7 times, and as well all know,
data *space* = I/O *time*.  In fact the better XML is even better than
that: when extended to a million products the space saving is better
than 1.95 and so is the time saving when parsing.

Does anyone remember that I proposed that Erlang could be
extended quite simply with XML expressions and XML patterns?

f() ->
     [  #product{sku=S,item=I,collection=C,Pages=P}
     || <'Export'>L</> <- xml:parse_file("my.xml")
      , <'Product' 'SKU'=S 'Item'=I 'Collection'=C 'Pages'=P/> <- L].

Without that extension, it would have to be something like

f() ->
     [  #product{sku=S,item=I,collection=C,Pages=P}
     || {'Export',_,L} <- xml:parse_file("my.xml")
      , {'Product',[{'Collection',C},{'Item',I},{'Pages',P},{'SKU',S}],
		  []} <- L].

except that this wouldn't work with additional attributes, and the
previous version would.

I also happen to think that good XML style imitates XHTML, SVG, MathML,
and other such standards in preferring lower case starts or even  
avoiding
upper case starts entirely, rather than imitating Visual Basic style.
Coincidentally that means rather less quoting in Erlang, so

f() ->
     [  #product{sku=S,item=I,collection=C,Pages=P}
     || {export,_,L} <- xml:parse_file("my.xml")
      , {product,[{'SKU',S},{collection,C},{item,I},{pages,P}], []}  
<- L].

This is doable *now* in Erlang, using an XML parser I wrote back in
June 2001.  I honestly cannot see this as inferior to the Ruby version.
So now let's revert to the inferior application of XML and see what
that looks like:

f() ->
     [  #product{sku=S,item=I,collection=C,Pages=P}
     || {'Export',_,L} <- xml:parse_file("my.xml")
      , {'Product',_,D} <- L
      , {'SKU',_,[S]} <- D
      , {'ItemName',_,[I]} <- D
      , {'CollectionNo',_,[C]} <- D
      , {'Pages',_,[P]} <- D
     ].

That wasn't *too* bad, was it?  Shorter than the Ruby version...

Handling namespaces is trickier, but thankfully, Erlang lets us
include *bound* variables in patterns, so we could do something like
this:

f() ->
     Main_NS = "http://www.example.org/silly/main",
     Attr_NS = "http://www.example.org/silly/attr",
     Export       = xml:name('Export',       Main_NS),
     Product      = xml:name('Product',      Main_NS),
     SKU          = xml:name('SKU',          Attr_NS),
     ItemName     = xml:name('ItemName',     Attr_NS),
     CollectionNo = xml:name('CollectionNo', Attr_NS),
     Pages        = xml:name('Pages',        Attr_NS),

     [  #product{sku=S,item=I,collection=C,Pages=P}
     || {Export,_,L} <- xml:parse_file("my.xml")
      , {Product,_,D} <- L
      , {SKU,_,[S]} <- D
      , {ItemName,_,[I]} <- D
      , {CollectionNo,_,[C]} <- D
      , {Pages,_,[P]} <- D
     ].

It doesn't get much easier than this, anywhere.  It so happens that
xml:name/2 doesn't exist.  It would be
	name(Name, NS) -> {Name,NS}.
but my old parser doesn't do namespaces.  Not much point in changing it
when there are more capable parsers around.

ML: doesn't allow bound variables in patterns, doesn't have list
     comprehension
Haskell and Clean: don't allow bound variables in patterns,
     do have list comprehension
Erlang: does allow bound variables in patterns, does have list
     comprehension, making this way of picking XML apart dead easy.





From per@REDACTED  Thu Oct 25 00:18:10 2007
From: per@REDACTED (Per Hedeland)
Date: Thu, 25 Oct 2007 00:18:10 +0200 (CEST)
Subject: [erlang-questions] Forming a cluster
In-Reply-To: <de422a40710241440o3b7da58cjeadabbdf13deadfb@mail.gmail.com>
Message-ID: <200710242218.l9OMIAai018605@pluto.hedeland.org>

"Kenneth Lundin" <kenneth.lundin@REDACTED> wrote:
>
>By using
>net_adm:names(Host)
>or
>net_adm:names()
>
>The newly started node can find out about other nodes already started.
>Of course this requires knowledge of the possible Hosts which can
>have running nodes.

It could also be claimed that this is "abuse" of epmd. Unless you have a
host/network dedicated to your application, you have no reason to assume
that you should be talking to other nodes that happen to have registered
with epmd on the hosts you're interested in - epmd's purpose in life is
to map "alive-names" to TCP distribution ports of currently running
nodes, nothing more.

I would rather recommend using "standard" IP techniques - e.g. designate
an "unused" UDP port as "point of contact" for your application. The
existing nodes in the cluster agree (somehow) among themselves who
should listen to that port on each host, and newly arriving nodes send a
"I'm here and my name is ..." message to the port. By using UDP you can
also use broadcast and/or (if you're lucky) multicast to find nodes on
other hosts.

--Per Hedeland


From bbmaj7@REDACTED  Thu Oct 25 00:20:08 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Thu, 25 Oct 2007 08:20:08 +1000 (EST)
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <471F4EC9.5030706@it.uu.se>
Message-ID: <270572.10302.qm@web52012.mail.re2.yahoo.com>


> > I currently use iolist_size() to check the size of the input binary then
> > recursively break off a chunk from the front with something like, eg.
> > 
> 
> Why are you using iolist_size()? Is it possiblethat your input is not a 
> binary?

What other way is there to determine the length of a binary?



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From joelr1@REDACTED  Thu Oct 25 01:13:40 2007
From: joelr1@REDACTED (Joel Reymont)
Date: Thu, 25 Oct 2007 00:13:40 +0100
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <270572.10302.qm@web52012.mail.re2.yahoo.com>
References: <270572.10302.qm@web52012.mail.re2.yahoo.com>
Message-ID: <CB0A7EDD-318E-4705-939C-10605256ABE0@gmail.com>


On Oct 24, 2007, at 11:20 PM, Richard Andrews wrote:

> What other way is there to determine the length of a binary?

Eshell V5.5.5  (abort with ^G)
1> size(<<>>).
0
2> size(<<1,2,3>>).
3

--
http://wagerlabs.com







From hokan.stenholm@REDACTED  Thu Oct 25 01:16:38 2007
From: hokan.stenholm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Stenholm?=)
Date: Thu, 25 Oct 2007 01:16:38 +0200
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <270572.10302.qm@web52012.mail.re2.yahoo.com>
References: <270572.10302.qm@web52012.mail.re2.yahoo.com>
Message-ID: <471FD256.1020405@bredband.net>

Richard Andrews wrote:
>>> I currently use iolist_size() to check the size of the input binary then
>>> recursively break off a chunk from the front with something like, eg.
>>>
>>>       
>> Why are you using iolist_size()? Is it possiblethat your input is not a 
>> binary?
>>     
>
> What other way is there to determine the length of a binary?
>   
size/1 (works on binaries and tuples), while length/1 works on lists e.g.:

 > size(<<>>).
0
 > size(<<"abcde">>).
5
 > size({}).
0
 > size({1,2,3}).
3

>
>
>       Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
> http://au.docs.yahoo.com/mail/unlimitedstorage.html
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>   


From ok@REDACTED  Thu Oct 25 01:38:06 2007
From: ok@REDACTED (Richard A. O'Keefe)
Date: Thu, 25 Oct 2007 12:38:06 +1300
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710241317o111cfb98j374e14c1beace83c@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
	<e00abaee0710241317o111cfb98j374e14c1beace83c@mail.gmail.com>
Message-ID: <D14E2C00-5154-4C11-A5A7-1B76F34E9DBC@cs.otago.ac.nz>

I get so incredibly frustrated when asked to crawl through a vast
trackless forest of pages.  I want to sit in a comfortable chair
under a good light and read nice crisp high contrast text; on paper!
Electronically, I want stuff I can *search*.  That argues for having
stuff in *big* pages where searching can get some traction.  Then
there are times when I want to find just one thing, and R's on-line
help (or for that matter, SWI Prolog's) is perfect.

The key point is that there is no *one* form which will serve all
common uses well.  One thing which is NOT a good idea is stuffing
on-line documentation chock full of Javascript.  A reader should
*NEVER* have to enable security holes (whoops, Javascript) in order
to get usable links.

One very important thing, by the way, is to remember that people do
need to print pages, and that not everyone has the same size paper,
and not everyone has the same size screen.

A very very important thing about documentation is that the javadoc
approach tends to encourage a very atomised view of documentation.
Overviews, models, examples, templates, all sorts of stuff are not
information about the single methods that Javadoc tends to be good at.

Tree navigation tools are great for monkeys.  For humans, a good
information retrieval engine is at least as important and doesn't
wipe out so much potentially useful screen real estate.



From dougedmunds@REDACTED  Thu Oct 25 02:31:46 2007
From: dougedmunds@REDACTED (Doug Edmunds)
Date: Wed, 24 Oct 2007 17:31:46 -0700
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
Message-ID: <f6e9854a0710241731i14fca7a7v4005db40762dcda7@mail.gmail.com>

If you use Windows, there are still free versions of X1 Enterprise Client
out there. (One place is from http://www.snapfiles.com/get/x1.html).  Have
it
index your erlang directory, then use it like a grep program,
It shows you the search results in the output format of the file
(including html, pdf). Click the link and it opens the file.

-dae
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071024/76d3305e/attachment.htm>

From jeffm@REDACTED  Thu Oct 25 02:50:09 2007
From: jeffm@REDACTED (jm)
Date: Thu, 25 Oct 2007 10:50:09 +1000
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710242022.l9OKMSr4015930@pluto.hedeland.org>
References: <200710242022.l9OKMSr4015930@pluto.hedeland.org>
Message-ID: <471FE841.7070200@ghostgun.com>

Per Hedeland wrote:

>>>> Is the port owner process still running? The port gets automatically
>>>> closed when the owner exits.
> 
> I.e. I'm referring to the Erlang process that opened the port (unless
> you have changed the port owner with e.g. port_connect/2). That process
> exiting is the most likely cause of the port getting closed from the
> Erlang side. Possibly the port could also be closed by the runtime if
> you have specifed some more "advanced" "protocol" than
> stream/packet/line and the incoming data doesn't follow it (any random
> data can be interpreted as being either of those three), but in that
> case the port owner should get an exit signal.


The erlang process which owns the port dies. I've checked to see that I 
didn't do something so that it's not tail recursive and it seems 
correct. Also checked to make sure I wasn't doing something stupid like 
printing debug info to stdout, checked that all messages sent to erlang 
are generated by ei and sent using write_cmd.

The port is created via

start() ->
     spawn(fun() ->
                   register(?MODULE, self()),
                   process_flag(trap_exit, true),
                   Port = open_port({spawn, "./inotify"},
                                    [{packet, 2}, binary, exit_status]),
                   loop(Port)
           end).

so nothing funny there. I wrote a stripped down port which just sat 
there and sent {ok, Count} messages once a second for 5 second where 
Count is the number of seconds minus one. This worked which turned my 
attention to getting more info out of erlang about what it was seeing.


Next, based on an email from fritchie@REDACTED Subject: Re: 
[erlang-questions] turning up debugging on ports,

4> dbg:tracer().
{ok,<0.38.0>}
5> dbg:p(all,call).
{ok,[{matched,nonode@REDACTED,25}]}
6> dbg:tpl(inotify,loop, [{'_',[],[{return_trace}]}]).
{ok,[{matched,nonode@REDACTED,1},{saved,1}]}
7> inotify:start().
<0.43.0>(<0.43.0>) call inotify:loop(#Port<0.98>)

8> inotify:open().
  ......
9> inotify:add(3, "file", all).
read_cmd start curpos = 0
read_cmd curpos = 2
read_cmd len = 24 desired=24
read_cmd result: 2, size: 100, cmdpos: 26
read_cmd returned message
add
atom = all
add: fd=3 pathname=file mask = FFF
wd = 1
note_read
note_read end while for fd = 3
note_read return
(<0.43.0>) call inotify:loop(#Port<0.98>)
call_port: {ok,1}
{ok,1}

%%% run "cat file" in shell

10> note_read
note_read fd = 3
encoding event wd=1, mask=32, cookie=0, len=0, name=
note_read send event
encoding event wd=1, mask=32, cookie=0, len=0, name=
note_read send event
note_read end while for fd = 3
note_read return 
    10> read_cmd start curpos = 0
read_cmd result: 0, size: 100, cmdpos: 0
read_cmd returned 0: Success(0)
10>

I'm not familar with the dbg module so I'm off to read the man page.

Jeff.


From jkakar@REDACTED  Thu Oct 25 03:09:26 2007
From: jkakar@REDACTED (Jamu Kakar)
Date: Wed, 24 Oct 2007 18:09:26 -0700
Subject: [erlang-questions] Rant: I hate parsing XML with Erlang
In-Reply-To: <6a36e7290710230830r32090986nddef00883c7f1f4d@mail.gmail.com>
References: <0A4E5EF3-C9BC-42EB-AABF-2F2DE009E38B@gmail.com>	<471DE3F8.3080903@volny.cz>	<7F502E24-4FED-4FE5-9078-72464E6C99BD@gmail.com>	<95be1d3b0710230602t7048e761ob87ae72f3026c9b2@mail.gmail.com>	<9708A0D0-C43F-4751-B365-CC3E49189FEB@gmail.com>	<54926545-2DF9-43D3-9E46-05652B29F6C9@gmail.com>	<09AE1952-AAEC-4D8B-876D-8B7F5E02D4F2@hypotheticalabs.com>	<93A84605-1197-4198-9731-5C32783DEBA8@gmail.com>	<A993E5B9-1770-4EF3-909F-6CDEB8AFCF12@hypotheticalabs.com>	<9b08084c0710230809w3f5e7c53uab617b088c19d99b@mail.gmail.com>
	<6a36e7290710230830r32090986nddef00883c7f1f4d@mail.gmail.com>
Message-ID: <471FECC6.5020906@kakar.ca>

Hi Bob,

Bob Ippolito wrote:
 > http://tidy.sourceforge.net/ is the typical library I've seen used to
 > transform arbitrary HTML into a valid document quickly and without
 > re-inventing the wheel. Much easier than trying to integrate with
 > Mozilla.

Tidy is good, maybe even the only workable solution, depending on your
needs.  It tries to convert malformed HTML into wellformed HTML that
you can parse with something that expects wellformed markup.  In
practice, it tends to be rather slow.  I had a need a few years ago to
parse arbitrary HTML.  I didn't care about making it wellformed; I
just needed something that could start at the beginning and raise
SAX-like events when it encountered stuff.  For example, the following
malformed document:

<html>
   <body>
     <p>I am an unclosed tag
   </body>
</html>

would result in:

start_tag: html
start_tag: body
start_tag: p
element_content: I am an unclosed tag
end_tag: body
end_tag: html

If this is good enough it might be worth looking at.  The code is part
of a C++ application framework I work on in my spare time.  You can
get the latest code at:

https://launchpad.net/framework

The relevant bits wrt to HTML parsing are here:

http://codebrowse.launchpad.net/~jkakar/framework/57186-release-0.2/files/jkakar%40starla-20070107232638-khamuslq9vz15z6a?file_id=text-20060821192009-a8646b9047718e3d

I've started porting the HTML parsing code to Python, but it's not
ready yet.  Maybe using this code would be helpful?  The code has been
used in a production environment and works and performs fairly well.

Thanks,
J.


From jeremie@REDACTED  Thu Oct 25 06:29:57 2007
From: jeremie@REDACTED (=?ISO-8859-1?Q?J=E9r=E9mie_Lumbroso?=)
Date: Thu, 25 Oct 2007 06:29:57 +0200
Subject: [erlang-questions] Proxy for URL getting
Message-ID: <2b7b425b0710242129t272c290fk381fe832727dc58a@mail.gmail.com>

Hello,

Is it possible and easy to download a link through a proxy? (More
specifically, is it possible to efficiently download RSS feeds through
Tor, the anonymous connection proxy?)

The www_tools seems to allow for a proxy, but I'm still trying to
figure out how to make it work.

Best Regards,

JL


From erlangx@REDACTED  Thu Oct 25 06:57:45 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Wed, 24 Oct 2007 21:57:45 -0700
Subject: [erlang-questions] Proxy for URL getting
In-Reply-To: <2b7b425b0710242129t272c290fk381fe832727dc58a@mail.gmail.com>
References: <2b7b425b0710242129t272c290fk381fe832727dc58a@mail.gmail.com>
Message-ID: <20071025045745.GH6096@delora.autosys.us>

 see http:set_options/1  

~Michael

On Thu, Oct 25, 2007 at 06:29:57AM +0200, J?r?mie Lumbroso wrote:
> Hello,
> 
> Is it possible and easy to download a link through a proxy? (More
> specifically, is it possible to efficiently download RSS feeds through
> Tor, the anonymous connection proxy?)
> 
> The www_tools seems to allow for a proxy, but I'm still trying to
> figure out how to make it work.
> 
> Best Regards,
> 
> JL
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,47201bf350987844086134!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From bbmaj7@REDACTED  Thu Oct 25 09:00:29 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Thu, 25 Oct 2007 17:00:29 +1000 (EST)
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <471FD256.1020405@bredband.net>
Message-ID: <398995.98160.qm@web52004.mail.re2.yahoo.com>


--- H?kan Stenholm <hokan.stenholm@REDACTED> wrote:

> > What other way is there to determine the length of a binary?
> >   
> size/1 (works on binaries and tuples), while length/1 works on lists e.g.:
> 
>  > size(<<>>).
> 0
>  > size(<<"abcde">>).
> 5
>  > size({}).
> 0
>  > size({1,2,3}).
> 3


Thanks.

I don't remember seeing this mentioned in any of the OTP docs.



      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From bjorn@REDACTED  Thu Oct 25 09:31:09 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 25 Oct 2007 09:31:09 +0200
Subject: [erlang-questions] Efficiency of splitting binaries
In-Reply-To: <398995.98160.qm@web52004.mail.re2.yahoo.com>
References: <398995.98160.qm@web52004.mail.re2.yahoo.com>
Message-ID: <o18x5rd51e.fsf@erix.ericsson.se>

Richard Andrews <bbmaj7@REDACTED> writes:

> --- H?kan Stenholm <hokan.stenholm@REDACTED> wrote:
> 
> I don't remember seeing this mentioned in any of the OTP docs.
> 

It is mentioned here:

http://www.erlang.org/doc/man/erlang.html#size/1

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From kenneth.lundin@REDACTED  Thu Oct 25 10:20:40 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Thu, 25 Oct 2007 10:20:40 +0200
Subject: [erlang-questions] Forming a cluster
In-Reply-To: <200710242218.l9OMIAai018605@pluto.hedeland.org>
References: <de422a40710241440o3b7da58cjeadabbdf13deadfb@mail.gmail.com>
	<200710242218.l9OMIAai018605@pluto.hedeland.org>
Message-ID: <de422a40710250120h165f0c03n207fea0d5b04e05f@mail.gmail.com>

I agree with Per.

I made the assumption that the hosts where dedicated for the clustered
application
and that no other Erlang nodes than the ones involved in the cluster should be
started on these hosts.

/Kenneth

On 10/25/07, Per Hedeland <per@REDACTED> wrote:
> "Kenneth Lundin" <kenneth.lundin@REDACTED> wrote:
> >
> >By using
> >net_adm:names(Host)
> >or
> >net_adm:names()
> >
> >The newly started node can find out about other nodes already started.
> >Of course this requires knowledge of the possible Hosts which can
> >have running nodes.
>
> It could also be claimed that this is "abuse" of epmd. Unless you have a
> host/network dedicated to your application, you have no reason to assume
> that you should be talking to other nodes that happen to have registered
> with epmd on the hosts you're interested in - epmd's purpose in life is
> to map "alive-names" to TCP distribution ports of currently running
> nodes, nothing more.
>
> I would rather recommend using "standard" IP techniques - e.g. designate
> an "unused" UDP port as "point of contact" for your application. The
> existing nodes in the cluster agree (somehow) among themselves who
> should listen to that port on each host, and newly arriving nodes send a
> "I'm here and my name is ..." message to the port. By using UDP you can
> also use broadcast and/or (if you're lucky) multicast to find nodes on
> other hosts.
>
> --Per Hedeland
>


From luna@REDACTED  Thu Oct 25 11:07:38 2007
From: luna@REDACTED (Daniel Luna)
Date: Thu, 25 Oct 2007 11:07:38 +0200 (CEST)
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
Message-ID: <Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>

On Wed, 24 Oct 2007, Kenneth Lundin wrote:
> I have plans to improve the doc's in the direction you ask for. I am 
> also considering using Javascript in a similar way as used in the 
> previously mentioned http://gotapi.com. But I think it is still a 
> requirement that the doc's can be downloaded and browsed locally without 
> the need for having a web-server running. I think it would be ok to 
> require a browser with javascript capabilities (we already require 
> javascript when it comes to links between applications)

And this javascript is actually quite irritating for me. I quite often 
wish to open links in a new tab, but when the link is a javscript thingie 
I only get an empty tab when I middle-click.

Please don't use them in the future version.

/Luna
-- 
Daniel Luna                           | Top reasons that I have a beard:
luna@REDACTED                     |  a) Laziness.
http://www.update.uu.se/~luna/        |  b) I can.
Don't look at my homepage (it stinks).|  c) I can get away with it.


From chsu79@REDACTED  Thu Oct 25 13:02:33 2007
From: chsu79@REDACTED (Christian S)
Date: Thu, 25 Oct 2007 13:02:33 +0200
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
	<Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
Message-ID: <bc5c855a0710250402p6dcab303x116b8d73216c4015@mail.gmail.com>

These problems are related to making links

<a href="javascript:open('/real/url/to/resource')">linktext</a>

instead of

<a href="/real/url/to/resource" onclick="return open(this.href);">linktext</a>

Everyone should do it the second way whenever the opened page has a
real url known at page-generation time.  Benefits are, as mentioned by
Luna,
that open-in-new-tab works. It's the web-2.0-way to surf. :)

PS. With js injection using prototypejs and jQuery one can do even
cleaner things, such as adding the onclick event to all links with a
certain class. But I have yet to experience all that much benefit from
such approaches.
(Thanks in advance for not starting a thread on the merits though :)

2007/10/25, Daniel Luna <luna@REDACTED>:
> And this javascript is actually quite irritating for me. I quite often
> wish to open links in a new tab, but when the link is a javscript thingie
> I only get an empty tab when I middle-click.
>
> Please don't use them in the future version.
>
> /Luna
> --
> Daniel Luna                           | Top reasons that I have a beard:
> luna@REDACTED                     |  a) Laziness.
> http://www.update.uu.se/~luna/        |  b) I can.
> Don't look at my homepage (it stinks).|  c) I can get away with it.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From saleyn@REDACTED  Thu Oct 25 14:27:59 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Thu, 25 Oct 2007 07:27:59 -0500
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
Message-ID: <47208BCF.2020807@gmail.com>

This would be very useful indeed.  I can suggest the following 
JavaScript library that significantly simplifies implementation many of 
the tasks you mentioned below:  http://www.jquery.com.

This library solves most of cross-browser compatibility issues, gives 
numerous higher order functions (filter, map, etc.), has built-in 
methods for exploring the DOM model using XPath and similar access 
methods, and much more.  It only requires inclusion of a reference to a 
single JavaScript file.

And here's a nice site for fast access to jquery documentation:
http://visualjquery.com

Regards,

Serge


Kenneth Lundin wrote:
> Hi again
> 
> I have plans to improve the doc's in the direction you ask for.
> I am also considering using Javascript in a similar way as used in the
> previously
> mentioned http://gotapi.com.
> But I think it is still a requirement that the doc's can be downloaded
> and browsed
> locally without the need for having a web-server running. I think it
> would be ok to
> require a browser with javascript capabilities (we already require
> javascript when it
> comes to links between applications)
> 
> The suggested approach in summary:
> - Stop using frames, Use CSS instead for the layout
> - Nicer fonts
> - Use Javascript to achieve expandable tree view navigation to the left
> - Search as you type support (also with Javascript) similar as gotapi.com
> ...
> 
> Any comments on this approach?
> 
> /Kenneth Erlang/OTP team at Ericsson
> 
> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
>> Ah, apologies for not being more specific.  I've been a Java programmer for
>> last 10 years, and so I was asking specificaly about documentation that was
>> more like Sun produces (or even the Ruby docs).  For example, it's nice to
>> have a link to all the methods at the top of the class/module instead of
>> scrolling down etc.
>>
>> I have been able to find answers to my questions via the api, I just want to
>> find them faster.
>>
>> Thanks.
>>
>> - Frank
>>
>>
>>
>> On 10/24/07, Kenneth Lundin <kenneth.lundin@REDACTED> wrote:
>>> Hi Frank,
>>>
>>> Please explain what you mean with "hyper linked" version and why you don't
>> as it
>>> seems, think that the available documentation on erlang.org
>>> http://erlang.org/doc/
>>>
>>> is hyperlinked?
>>>
>>> We are well aware of that the html format is not modern and that it
>>> would be good with better search facilities but I don't know if that
>>> is what you are after.
>>>
>>> /Kenneth Erlang/OTP team at Ericsson
>>>
>>> On 10/24/07, Frank Blecha <frank.blecha@REDACTED> wrote:
>>>> Is there a hyper linked version out there somewhere?  I have been using
>> the
>>>> module docs from erlang.org, but a linked version would be a bit
>> handier.
>>>> Thanks.
>>>>
>>>> - Frank
>>>>
>>>> --
>>>> Frank Blecha
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED
>>>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>>>
>>
>>
>> --
>> Frank Blecha
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 



From michael.campbell@REDACTED  Thu Oct 25 13:03:21 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Thu, 25 Oct 2007 07:03:21 -0400
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
	<Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
Message-ID: <811f2f1c0710250403w4e8d7219l5e6dda0ee3226ebf@mail.gmail.com>

As a general rule I agree.  Links should be links.

On 10/25/07, Daniel Luna <luna@REDACTED> wrote:
> On Wed, 24 Oct 2007, Kenneth Lundin wrote:
> > I have plans to improve the doc's in the direction you ask for. I am
> > also considering using Javascript in a similar way as used in the
> > previously mentioned http://gotapi.com. But I think it is still a
> > requirement that the doc's can be downloaded and browsed locally without
> > the need for having a web-server running. I think it would be ok to
> > require a browser with javascript capabilities (we already require
> > javascript when it comes to links between applications)
>
> And this javascript is actually quite irritating for me. I quite often
> wish to open links in a new tab, but when the link is a javscript thingie
> I only get an empty tab when I middle-click.
>
> Please don't use them in the future version.
>
> /Luna
> --
> Daniel Luna                           | Top reasons that I have a beard:
> luna@REDACTED                     |  a) Laziness.
> http://www.update.uu.se/~luna/        |  b) I can.
> Don't look at my homepage (it stinks).|  c) I can get away with it.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From dcaoyuan@REDACTED  Thu Oct 25 15:12:28 2007
From: dcaoyuan@REDACTED (Caoyuan)
Date: Thu, 25 Oct 2007 21:12:28 +0800
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710240500k7a178721pf53aa591f766a633@mail.gmail.com>
References: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
	<417604.65860.qm@web38806.mail.mud.yahoo.com>
	<65b2728e0710240500k7a178721pf53aa591f766a633@mail.gmail.com>
Message-ID: <ca96bd270710250612x23c0b576maf9fe9c2e7dbf06c@mail.gmail.com>

It seems heap size is really a key for binary processing, and there
are other tips for binary processing too. With proper heap size set,
the straightforward Erlang code (in 80 LOC) can achieve around 3.1 sec
on my 4-CPU linux box (the ruby code took about 4.1 sec on the same
machine). The code is pasted on:

http://blogtrader.net/page/dcaoyuan/entry/learning_coding_binary_was_tim

With default heap size, the code may take 4.8+ sec.

On 10/24/07, Steve Vinoski <vinoski@REDACTED> wrote:
>
> On 10/24/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
> >
> > --- Steve Vinoski <vinoski@REDACTED> wrote:
> >
> > Anders, thanks for collecting and posting these.
> > > I've just performed a set
> > > of new timings for all of them, as listed below. For
> > > each, I just ran this
> > > command:
> > >
> > > time erl -smp -noshell -run <test_case> main
> > > o1000k.ap >/dev/null
> > >
> > > where "<test_case>" is the name of the tbray test
> > > case file. All were
> > > looped ten times, and I took the best timing for
> > > each. All tests were done
> > > on my 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM
> > > Linux box, in a local
> > > (non-NFS) directory.
> >
> > So, looking at Steve's results on his 8-core system,
> > we have:
> >
> >             real    user     tbray5/real    user/real
> > tbray5      9.8     --       1.0            --
> > tbray14     6.63    34.53    1.48            5.21
> > tbray15     4.12    25.14    2.38           6.10
> > tbray16     3.16    16.15    3.10           5.11
> > tbray_tuple 2.28     8.61    4.30           3.78
> > tbray_ets   1.87     7.42    5.24           3.97
> > tbray_blkr  1.55     7.34    6.32           4.74
> >
> > tbray5/real is the speedup versus the baseline, while
> > user/real is the speedup for each version due to
> > parallelization.
> >
> > Thus, the latest version is 6.3 times faster than the
> > first one. The parallel speedup is about the same in
> > tbray5 and tbray_blkr, a very decent utilization of
> > >50%, but the amount of work (user) has shrunk from
> > (presumably more than) 34.53 seconds to 7.34 seconds.
> >
> > Tim Bray's original Erlang number on "his macbook"
> > appears to be 34.16 seconds user (probably about the
> > same real?). How does this compare to Ruby? Tim Bray
> > reported that it needed 3.46 seconds real, again on
> > his macbook. (As I understand it, all results here are
> > for the big data set.)
> >
>
> Yes, all results are for o1000k.ap, Tim's original large dataset. As for
> Ruby, I just ran Tim's original code on the 8-core, and out of ten attempts
> the best was:
>
>
> real    0m2.210s
> user    0m2.095s
> sys     0m0.109s
> --steve
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
- Caoyuan


From Woolla_T@REDACTED  Thu Oct 25 16:08:09 2007
From: Woolla_T@REDACTED (Trevor Woollacott [ MTN - Innovation Centre ])
Date: Thu, 25 Oct 2007 16:08:09 +0200
Subject: [erlang-questions] Unexpected error report from Global name server
Message-ID: <70D00C33FCD1FD4A860DEAC228277C0C0309A2D7@MTNMAIL1.mtn.co.za>

Hi

 

I'm using Erlang R11-B4.

 

I started 2 runtimes using -connect_all false

and get error reports when I connect the nodes.

 

Node1:

Erlang (BEAM) emulator version 5.5.4 [async-threads:0]

 

Eshell V5.5.4  (abort with ^G)

(admin4@REDACTED)1> 

=ERROR REPORT==== 24-Oct-2007::15:13:20 ===

The global_name_server received an unexpected message:

handle_info({nodeup,admin3@REDACTED}, _)

 

 

Node2:

Erlang (BEAM) emulator version 5.5.4 [async-threads:0]

 

Eshell V5.5.4  (abort with ^G)

(admin3@REDACTED)1> auth:is_auth( admin4@REDACTED ).

 

=ERROR REPORT==== 24-Oct-2007::15:13:20 ===

The global_name_server received an unexpected message:

handle_info({nodeup,admin4@REDACTED}, _)

yes

 

 

I tested the same on R11-B2, and no error reports are generated.

 

Is this an intended feature?

 

 

Regards

Primanathan Reddy



NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/default.aspx?pid=34411 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/940de12f/attachment.htm>

From andreas.hillqvist@REDACTED  Thu Oct 25 16:17:03 2007
From: andreas.hillqvist@REDACTED (Andreas Hillqvist)
Date: Thu, 25 Oct 2007 16:17:03 +0200
Subject: [erlang-questions] Construct a function refrence from variables
Message-ID: <8268eea30710250717m2c8ed452l29035616d5750daa@mail.gmail.com>

In your case when the number of arguments is known, just pass the name
of the module and call it with:

     1> Mod = io.
     io
     2> Mod:format("test~n").

This is how call backs for gen_server is called.
But it requires that the receiver knows the name of the function.

But using this principle you can pass a the modules nme as an atom, or
a both module and functions name as a tuple:

     do(Fun) when is_function(Fun) ->
         Data = "FooBar",
         Fun(Data);
     do(Module) when is_atom(Module) ->
         Data = "FooBar",
         Module:format(Data);
     do({Module, Function}) ->
         Data = "FooBar",
         Module:Function(Data).


Or you can make this into a fun:

     1> Mod = io.
     io
     2>Fun = fun(Data) -> Mod:format(Data) end.


But fun's are slower then the Module:Function(Data) call.


Regards
Andreas Hillqvist

________________________________

From: William FINK
Sent: den 25 oktober 2007 13:28
To: TrapExit
Subject: [erlang-questions] Construct a function refrence from variables

If I need a function reference, I use this syntax:

> Fun = fun io:format/1.
#Fun<io.format.1>

> Fun("test~n").
test
ok

Now, I want to construct my function reference from a module name:

> Mod = io.
io
1> Fun2 = Mod:format/1.
** 1: illegal expression **

The problem is that I need to call a function from an unknown module quite a lot
Calling apply each time will be slower than using a function reference.

How can I construct a function reference from variables ?

_________________
William FINK


From nick@REDACTED  Thu Oct 25 16:29:51 2007
From: nick@REDACTED (Niclas Eklund)
Date: Thu, 25 Oct 2007 16:29:51 +0200 (MEST)
Subject: [erlang-questions] Bidirectional GIOP
In-Reply-To: <20071024215715.U1567@olgeni.olgeni>
Message-ID: <Pine.GSO.4.10.10710251004290.795-100000@shagrat>


Hello!

The ORB_ENV_USE_BI_DIR_IIOP flag is currently ignored, which is why it
hasn't been documented.

As you might know, GIOP messages can be framented. The FragmenHeader_1_2
struct only contains the request_id (GIOP 1.2), which makes it impossible
to know if the fragment belongs to a Request or Reply if Bidirectional
IIOP is allowed. Ok, this requires that both ORB:s concurrently has an
active request assigned the same request number. The risk is very slim,
but if both ORB:s support the Fault Tolerant specification that problem is
solved. Since an ORB must be able to handle fragmented GIOP messages and
can choose to ignore the Bidirectional service context, supporting
fragmented messages is more important.

The advantage with Bidirectional IIOP is that it might be easier to
configure a firewall, but since Orber supports a number of configuration
parameters that allows you to narrow the interfaces and ports that will be
used, this feature isn't very important. See, for example, the
configuration parameters iiop_out_ports, ip_address and ip_address_local.
You should also read the chapters 'Firewall Configuration' and 'Interface
Configuration' in Orber's User's Guide.

/Niclas


On Wed, 24 Oct 2007, Giacomo Olgeni wrote:

> Hello,
> 
> I was trying to activate bidirectional GIOP in Orber and found a
> ORB_ENV_USE_BI_DIR_IIOP flag in orber_iiop.hrl, however it is not
> documented in the Orber guide. Is it considered safe for production
> use?
> 
> Kind regards,
>    Jimmy
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 











From bjarne@REDACTED  Thu Oct 25 15:57:51 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Thu, 25 Oct 2007 15:57:51 +0200
Subject: [erlang-questions] Erlang User Conference 2007 - No seats left -
	Conference is fully booked
References: <290b3ba10710091534p30e8dfcrd3766b42439e8b64@mail.gmail.com>	<e3e496f90710091842k60628086oe382cf1d4b3afc8c@mail.gmail.com><290b3ba10710091852l72201aebib31eec3a3f125c30@mail.gmail.com>
	<470C892B.2070502@cs.chalmers.se>
Message-ID: <006101c8170f$20bcc6d0$c61669d4@Dell>

Dear Erlang enthusiasts

I am sorry to inform you all that this year's Erlang User Conference

http://www.erlang.se/euc/07/

is now fully booked.

Interest in the conference has been unprecedented. About 100 persons had 
registered based on the call-for-papers only.

Next year we will have to organise a larger lecture hall.

Best wishes

Bjarne



From jay@REDACTED  Thu Oct 25 16:44:59 2007
From: jay@REDACTED (Jay Nelson)
Date: Thu, 25 Oct 2007 07:44:59 -0700
Subject: [erlang-questions] Intro to Erlang Exercise #1 (Was OO -> FP)
Message-ID: <B8D0F780-F1C8-4FC0-9E7F-714227723267@duomark.com>

Recently there were questions about grokking erlang when coming from  
OO and how to design a system.  Here is an exercise to demonstrate  
some introductory erlang techniques:

1) Do the standard polygon problem by implementing a module with a  
record and a set of functions for manipulating polygons.

-module(polygon).
-record(shape, { ... }.

new(triangle) ->
    #shape{ ... };
new(rectangle) ->
    #shape{ ... };
new(square) ->
    #shape{ ... }.

delete(#shape{}) ->
    undefined.



Start out with a small set like triangle, rectangle and square before  
adding more.   Add a couple functions like perimeter and area.

2) Using the erlang shell, bind a few variables with new polygons and  
compare the results of calling the different functions.  Inspect the  
resulting records and notice how you can keep an old version around  
and compare it to a newer version.

3) Upgrade the implementation to include locations of vertices and  
the ability to move, compute center point and distance between  
centers of two polygons.

4) Add a second module which organizes a collection of polygons into  
town square with a central rectangular road and various shaped  
buildings at different addresses on the road.


This exercise demonstrates the following things about erlang:

1) Module is the fundamental form of code encapsulation
2) Records are good places to store structured data, learn how to use  
their syntax
3) Functions as transformers from old data instance to new data instance
4) Modules + records create an equivalence to getters / setters and  
manipulators in OO
5) The shell provides a dynamic way to test code and inspect structures
6) Direct translation of OO type instances is quite simple
7) How to code, debug, compile, run, etc.
8) Module composition can be used to add complexity on top of  
existing code but at a higher level of abstraction

Key understandings when comparing to OO:

1) Encapsulation is similar, but records are not opaque
2) Static data manipulation modules are useful basic abstractions  
allowing easy enhancement
3) Behavior is separate from data structure and data maintenance
4) Complex system architecture decisions can be deferred more easily


The exercise creates a representation of data instances plus  
manipulators and a coordinated higher level application.  It does not  
make any decisions about concurrency, persistence, distribution,  
versioning of instances, etc.   These are all behaviors which can be  
added by later modules -- in fact it is possible to have one or more  
modules which have differing implementations of these architectural  
decisions and have the system dynamically switch among the solutions  
as needed, all the while using the same underlying data structure  
representation.


For a bonus consider how you might solve the following problem:

1) Create a process that has a town center described
2) Enhance the polygon module to include color
3) Dynamically upgrade the process to include colors on buildings
       a) You could replace the old process with a new one
       b) You could replace the town center elements directly inside  
the original process


jay



From frank.blecha@REDACTED  Thu Oct 25 16:58:30 2007
From: frank.blecha@REDACTED (Frank Blecha)
Date: Thu, 25 Oct 2007 08:58:30 -0600
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <811f2f1c0710250403w4e8d7219l5e6dda0ee3226ebf@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
	<Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
	<811f2f1c0710250403w4e8d7219l5e6dda0ee3226ebf@mail.gmail.com>
Message-ID: <e00abaee0710250758i58c069d1odd2a7abfbf6f8258@mail.gmail.com>

Maybe we should refocus on documentation instead of javascript?

On 10/25/07, Michael Campbell <michael.campbell@REDACTED> wrote:
>
> As a general rule I agree.  Links should be links.
>
> On 10/25/07, Daniel Luna <luna@REDACTED> wrote:
> > On Wed, 24 Oct 2007, Kenneth Lundin wrote:
> > > I have plans to improve the doc's in the direction you ask for. I am
> > > also considering using Javascript in a similar way as used in the
> > > previously mentioned http://gotapi.com. But I think it is still a
> > > requirement that the doc's can be downloaded and browsed locally
> without
> > > the need for having a web-server running. I think it would be ok to
> > > require a browser with javascript capabilities (we already require
> > > javascript when it comes to links between applications)
> >
> > And this javascript is actually quite irritating for me. I quite often
> > wish to open links in a new tab, but when the link is a javscript
> thingie
> > I only get an empty tab when I middle-click.
> >
> > Please don't use them in the future version.
> >
> > /Luna
> > --
> > Daniel Luna                           | Top reasons that I have a beard:
> > luna@REDACTED                     |  a) Laziness.
> > http://www.update.uu.se/~luna/        |  b) I can.
> > Don't look at my homepage (it stinks).|  c) I can get away with it.
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
Frank Blecha
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/d79c9b4b/attachment.htm>

From erlang@REDACTED  Thu Oct 25 17:13:00 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Thu, 25 Oct 2007 17:13:00 +0200
Subject: [erlang-questions] GUID
Message-ID: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>

Does anybody have some Erlang code to generate a GUID?
(or to read the MAC address of the local machine interfaces?)

Thanks

/Joe


From dcaoyuan@REDACTED  Thu Oct 25 17:24:50 2007
From: dcaoyuan@REDACTED (Caoyuan)
Date: Thu, 25 Oct 2007 23:24:50 +0800
Subject: [erlang-questions] GUID
In-Reply-To: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
Message-ID: <ca96bd270710250824q72069bdcge7a687eda43668a3@mail.gmail.com>

I have piece of code, but with no function to read the MAC address of
the local machine interfaces:


-define(NUM_CHR_UPPER,
{$0,$1,$2,$3,$4,$5,$6,$7,$8,$9,$A,$B,$C,$D,$E,$F,$G,$H,$I,$J,$K,$L,$M,$N,$O,$P,$Q,$R,$S,$T,$U,$V,$W,$X,$Y,$Z}).
-define(NUM_CHR_LOWER,
{$0,$1,$2,$3,$4,$5,$6,$7,$8,$9,$a,$b,$c,$d,$e,$f,$g,$h,$i,$j,$k,$l,$m,$n,$o,$p,$q,$r,$s,$t,$u,$v,$w,$x,$y,$z}).


number_to_based_str(Number, Base) -> number_to_based_str(Number, Base, []).
number_to_based_str(Number, Base, Digits) when is_integer(Digits) ->
    Str = number_to_based_str(Number, Base, []),
    case Digits - length(Str) of
        Rem when Rem > 0 -> lists:duplicate(Rem, $0) ++ Str;
        _                -> Str
    end;
number_to_based_str(Number, Base, _Acc) when Number < Base ->
    [element(Number + 1, ?NUM_CHR_LOWER)];
number_to_based_str(Number, Base,  Acc) ->
    Msd = Number div Base,
    Lsd = Number rem Base,
    case Msd >= Base of
        true  -> number_to_based_str(Msd, Base, [element(Lsd + 1,
?NUM_CHR_LOWER)|Acc]);
        false -> [element(Msd + 1, ?NUM_CHR_LOWER)|[element(Lsd + 1,
?NUM_CHR_LOWER)|Acc]]
    end.

%% @doc Generates a GUID per http://www.ietf.org/rfc/rfc4122.txt
(version 1 generator)
gen_guid() ->
    TimeIn100NanosBeg = calendar:datetime_to_gregorian_seconds({{1582,
10, 15}, {0, 0, 0}}) * 10000000,
    {_MegaSecs, _Secs, MicroSecs} = now(),
    TimeIn100NanosNow =
calendar:datetime_to_gregorian_seconds(calendar:universal_time()) *
10000000 + MicroSecs * 10,
    Time = TimeIn100NanosNow - TimeIn100NanosBeg,
    TimeStr = number_to_based_str(Time, 16, 2),
    TimeStr1 =
      case 15 - length(TimeStr) of
          Rem when Rem > 0 -> lists:duplicate(Rem, $0) ++ TimeStr;
          _                -> TimeStr
      end,
    TimeHiV = lists:sublist(TimeStr1, 1, 3) ++ "1", %% add version number 1,
    TimeMid = lists:sublist(TimeStr1, 4, 4),
    TimeLow = lists:sublist(TimeStr1, 8, 8),
    ClockSeqHiV = io_format("~2.16.0b", [(random:uniform(256) - 1)
band 16#3f bor 16#80]), %% multiplexed variant type (2 bits)
    ClockSeqLow = io_format("~2.16.0b", [(random:uniform(256) - 1)]),
    Node = "001b631ee26b", %% an Ethernet MAC
    TimeLow ++ "-" ++ TimeMid ++ "-" ++ TimeHiV ++ "-" ++ ClockSeqHiV
++ ClockSeqLow ++ "-" ++ Node.

%% @doc Generates a random GUID per
http://www.ietf.org/rfc/rfc4122.txt (version 4 generator)
%% e.g. output: 372472a2-d557-4630-bc7d-bae54c934da1
%% word*2-, word-, (w)ord-, (w)ord-, word*3
gen_guid_v4() ->
    lists:foldl(
      fun (I, Acc) ->
              B = random:uniform(256) - 1, %% return random number
between 0, 255
              S = if  I == 7 -> %% multiplex version number (4 bits)
                          B1 = B band 16#0f bor 16#40, %% version 4 (random)
                          %% The last 0 in 2.16.0 means fill with
leading 0 if necessay
                          io_format("~2.16.0b", [B1]);
                      I == 9 -> %% multiplexed variant type (2 bits)
                          B1 = B band 16#3f bor 16#80,
                          io_format("~2.16.0b", [B1]);
                      I == 4; I == 6; I == 8; I == 10 ->
                          io_format("~2.16.0b-", [B]);
                      true ->
                          io_format("~2.16.0b", [B])
                  end,
              Acc ++ S
      end, [], lists:seq(1, 16)).

On 10/25/07, Joe Armstrong <erlang@REDACTED> wrote:
> Does anybody have some Erlang code to generate a GUID?
> (or to read the MAC address of the local machine interfaces?)
>
> Thanks
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
- Caoyuan


From taavi@REDACTED  Thu Oct 25 17:58:48 2007
From: taavi@REDACTED (Taavi Talvik)
Date: Thu, 25 Oct 2007 18:58:48 +0300
Subject: [erlang-questions] GUID
In-Reply-To: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
Message-ID: <435AB503-84E5-48A1-AA9E-1C3686378EBE@uninet.ee>


On Oct 25, 2007, at 6:13 PM, Joe Armstrong wrote:

> Does anybody have some Erlang code to generate a GUID?
> (or to read the MAC address of the local machine interfaces?)

Probably machine/os specific, on freebsd you can do

1> os:cmd("sysctl kern.hostuuid").
"kern.hostuuid: B5D5342C-9142-3AA0-81C8-7391B34506DA\n"


best regards,
taavi


From puzza007@REDACTED  Thu Oct 25 19:15:00 2007
From: puzza007@REDACTED (Paul Oliver)
Date: Thu, 25 Oct 2007 18:15:00 +0100
Subject: [erlang-questions] call_port with binary
Message-ID: <a4cc77ec0710251015n3fd14655p60a77412f60aefdf@mail.gmail.com>

Hi all,

I have a piece of code that does:

port_command(Port, term_to_binary(Msg)),

where msg is:

{Num, Binary}

This message never seems to get to my port, whereas other messages
with non-binary args in the tuple do.  Does anyone know why this could
be?

Thanks in advance,
Paul.


From rsaccon@REDACTED  Thu Oct 25 19:25:53 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 25 Oct 2007 14:25:53 -0300
Subject: [erlang-questions] GUID
In-Reply-To: <435AB503-84E5-48A1-AA9E-1C3686378EBE@uninet.ee>
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
	<435AB503-84E5-48A1-AA9E-1C3686378EBE@uninet.ee>
Message-ID: <eb4177b00710251025j559ab5dr7b1d5479397585c2@mail.gmail.com>

Something like this gives me the MAC address on OS X;

os:cmd("ipconfig getpacket en0 | grep chaddr | awk '{print $3}'").


On 10/25/07, Taavi Talvik <taavi@REDACTED> wrote:
>
> On Oct 25, 2007, at 6:13 PM, Joe Armstrong wrote:
>
> > Does anybody have some Erlang code to generate a GUID?
> > (or to read the MAC address of the local machine interfaces?)
>
> Probably machine/os specific, on freebsd you can do
>
> 1> os:cmd("sysctl kern.hostuuid").
> "kern.hostuuid: B5D5342C-9142-3AA0-81C8-7391B34506DA\n"
>
>
> best regards,
> taavi
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Roberto Saccon
http://rsaccon.com


From yinso.chen@REDACTED  Thu Oct 25 19:19:12 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 10:19:12 -0700
Subject: [erlang-questions] erlang api documentation
In-Reply-To: <e00abaee0710250758i58c069d1odd2a7abfbf6f8258@mail.gmail.com>
References: <e00abaee0710240854x3994af57v165a50ed2cba3479@mail.gmail.com>
	<de422a40710241003o7d991b13u9f19a22c89542dc3@mail.gmail.com>
	<e00abaee0710241153o2b475271y93e0b8a0a252fe0c@mail.gmail.com>
	<de422a40710241241obc63afdnee25b662ddb85e17@mail.gmail.com>
	<Pine.LNX.4.64.0710251051040.13522@Psilocybe.Update.UU.SE>
	<811f2f1c0710250403w4e8d7219l5e6dda0ee3226ebf@mail.gmail.com>
	<e00abaee0710250758i58c069d1odd2a7abfbf6f8258@mail.gmail.com>
Message-ID: <779bf2730710251019x7462ef42ref8a7dad9e544864@mail.gmail.com>

My 2 cents,

1) is Erlang documentation consumable?  i.e. XML or some other manipulatable
format.  If so, the user interface just need to be adequate.  People can
always import into their favorite doc browser or print.

2) Erlang's documentation is currently sparse, compared to perl, php,
python, etc.  A suggestion is to "open up" the documentation and have user
help improve that, like wiki style or comment style such as php/postgresql
(or there might be better ideas out there).


On 10/25/07, Frank Blecha <frank.blecha@REDACTED> wrote:
>
> Maybe we should refocus on documentation instead of javascript?
>
> On 10/25/07, Michael Campbell <michael.campbell@REDACTED > wrote:
> >
> > As a general rule I agree.  Links should be links.
> >
> > On 10/25/07, Daniel Luna < luna@REDACTED> wrote:
> > > On Wed, 24 Oct 2007, Kenneth Lundin wrote:
> > > > I have plans to improve the doc's in the direction you ask for. I am
> > > > also considering using Javascript in a similar way as used in the
> > > > previously mentioned http://gotapi.com. But I think it is still a
> > > > requirement that the doc's can be downloaded and browsed locally
> > without
> > > > the need for having a web-server running. I think it would be ok to
> > > > require a browser with javascript capabilities (we already require
> > > > javascript when it comes to links between applications)
> > >
> > > And this javascript is actually quite irritating for me. I quite often
> >
> > > wish to open links in a new tab, but when the link is a javscript
> > thingie
> > > I only get an empty tab when I middle-click.
> > >
> > > Please don't use them in the future version.
> > >
> > > /Luna
> > > --
> > > Daniel Luna                           | Top reasons that I have a
> > beard:
> > > luna@REDACTED                     |  a) Laziness.
> > > http://www.update.uu.se/~luna/ <http://www.update.uu.se/%7Eluna/>        |  b)
> > I can.
> > > Don't look at my homepage (it stinks).|  c) I can get away with it.
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
>
> --
> Frank Blecha
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/31e82176/attachment.htm>

From per@REDACTED  Thu Oct 25 22:35:46 2007
From: per@REDACTED (Per Hedeland)
Date: Thu, 25 Oct 2007 22:35:46 +0200 (CEST)
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <471FE841.7070200@ghostgun.com>
Message-ID: <200710252035.l9PKZkun046014@pluto.hedeland.org>

jm <jeffm@REDACTED> wrote:
>
>Per Hedeland wrote:
>
>>>>> Is the port owner process still running? The port gets automatically
>>>>> closed when the owner exits.
>> 
>> I.e. I'm referring to the Erlang process that opened the port (unless
>> you have changed the port owner with e.g. port_connect/2). That process
>> exiting is the most likely cause of the port getting closed from the
>> Erlang side. Possibly the port could also be closed by the runtime if
>> you have specifed some more "advanced" "protocol" than
>> stream/packet/line and the incoming data doesn't follow it (any random
>> data can be interpreted as being either of those three), but in that
>> case the port owner should get an exit signal.
>
>
>The erlang process which owns the port dies.

So there you have it, now you just need to figure out *why* it dies -
no need to try low-level port I/O debugging for that.

> I've checked to see that I 
>didn't do something so that it's not tail recursive and it seems 
>correct. Also checked to make sure I wasn't doing something stupid like 
>printing debug info to stdout, checked that all messages sent to erlang 
>are generated by ei and sent using write_cmd.

This is sort-of irrelevant - if the port was closed by the VM "for some
reason", your process would get an exit signal that might kill it, but
since you're trapping exits it won't. And forget what I wrote about
"advanced protocols", there aren't any for an external-process port
(temporary synapse crosstalk delivered some inet driver functionality) -
i.e. the data that arrives on such a port from the outside can never be
"wrong" from the VM's point of view.

I can however be wrong from your process' point of view - e.g. if it
expects to receive binaries in Erlang external term format, and
optimistically does a binary_to_term/1 on whatever it gets, it will die
with a badarg if your port program sends garbage - and the port will be
closed *as a result of your process dying*. At least during debugging
you may want to wrap catch and/or try around such things. And at least
when running from the shell, you should get a message about the
immediate cause of any "unnatural" death:

=ERROR REPORT==== 25-Oct-2007::22:30:11 ===
Error in process <0.52.0> with exit value: {badarg,[{erlang,binary_to_term,[<<3 bytes>>]},{ll,loop,1}]}


--Per


From erlang@REDACTED  Thu Oct 25 22:46:41 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Thu, 25 Oct 2007 22:46:41 +0200
Subject: [erlang-questions] disk merging
Message-ID: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>

I have an interesting? problem.

Over the last ? years (> 10) I have been upgrading my home system
this usually involved buying a bigger disk and copying most (or all)
of the files
from the old disk to the new disk or disks.

I've also been backing up the family photos etc on USB disks.

Now I have > 1 Tera bytes of files spread over c. 10 computers and
3 pluggable USB disks. Having made a "backup" both the original and the
copy live lives of their own.

Does anybody know of a good algorithm to consolidate/merge all this
data or do I have
to write my own? One immediate thought is to compute the MD5 sums of
all files on all
disk and thus find all duplicates - then create a master copy of all
unique files
but the file names will be wrong and this might result in a big mess.

This cannot be an uncommon problem - any ideas how to solve it?

/Joe


From tali.wang@REDACTED  Thu Oct 25 23:07:16 2007
From: tali.wang@REDACTED (Linan Wang)
Date: Thu, 25 Oct 2007 22:07:16 +0100
Subject: [erlang-questions] disk merging
In-Reply-To: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
Message-ID: <abb391f80710251407k5e8c157blc80f3b1070e1aba1@mail.gmail.com>

Interesting problem.
It appears to me like a svn/cvs/git related problem. I think the core
is to merge indices. Have a look at git, in my memory it has such kind
of function.
Hopefully Time Machine will solve this kind of problems.

On 25/10/2007, Joe Armstrong <erlang@REDACTED> wrote:
> I have an interesting? problem.
>
> Over the last ? years (> 10) I have been upgrading my home system
> this usually involved buying a bigger disk and copying most (or all)
> of the files
> from the old disk to the new disk or disks.
>
> I've also been backing up the family photos etc on USB disks.
>
> Now I have > 1 Tera bytes of files spread over c. 10 computers and
> 3 pluggable USB disks. Having made a "backup" both the original and the
> copy live lives of their own.
>
> Does anybody know of a good algorithm to consolidate/merge all this
> data or do I have
> to write my own? One immediate thought is to compute the MD5 sums of
> all files on all
> disk and thus find all duplicates - then create a master copy of all
> unique files
> but the file names will be wrong and this might result in a big mess.
>
> This cannot be an uncommon problem - any ideas how to solve it?
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Best regards

Linan Wang


From harveyd@REDACTED  Thu Oct 25 23:25:38 2007
From: harveyd@REDACTED (Dale Harvey)
Date: Thu, 25 Oct 2007 22:25:38 +0100
Subject: [erlang-questions] disk merging
In-Reply-To: <abb391f80710251407k5e8c157blc80f3b1070e1aba1@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
	<abb391f80710251407k5e8c157blc80f3b1070e1aba1@mail.gmail.com>
Message-ID: <fc6cbe440710251425v193f0d43g755a9764a243a1be@mail.gmail.com>

I actually read a recent article on planet erlang about locating duplicate
files
that has some information that might help

http://planeterlang.org/story.php?title=Writing_a_duplicate_file_finder_in_Erlang

On 25/10/2007, Linan Wang <tali.wang@REDACTED> wrote:
>
> Interesting problem.
> It appears to me like a svn/cvs/git related problem. I think the core
> is to merge indices. Have a look at git, in my memory it has such kind
> of function.
> Hopefully Time Machine will solve this kind of problems.
>
> On 25/10/2007, Joe Armstrong <erlang@REDACTED> wrote:
> > I have an interesting? problem.
> >
> > Over the last ? years (> 10) I have been upgrading my home system
> > this usually involved buying a bigger disk and copying most (or all)
> > of the files
> > from the old disk to the new disk or disks.
> >
> > I've also been backing up the family photos etc on USB disks.
> >
> > Now I have > 1 Tera bytes of files spread over c. 10 computers and
> > 3 pluggable USB disks. Having made a "backup" both the original and the
> > copy live lives of their own.
> >
> > Does anybody know of a good algorithm to consolidate/merge all this
> > data or do I have
> > to write my own? One immediate thought is to compute the MD5 sums of
> > all files on all
> > disk and thus find all duplicates - then create a master copy of all
> > unique files
> > but the file names will be wrong and this might result in a big mess.
> >
> > This cannot be an uncommon problem - any ideas how to solve it?
> >
> > /Joe
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> --
> Best regards
>
> Linan Wang
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>



-- 
http://arandomurl.com/
http://www.flickr.com/photos/daleharvey/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/4f450c34/attachment.htm>

From harveyd@REDACTED  Thu Oct 25 23:28:16 2007
From: harveyd@REDACTED (Dale Harvey)
Date: Thu, 25 Oct 2007 22:28:16 +0100
Subject: [erlang-questions] disk merging
In-Reply-To: <fc6cbe440710251425v193f0d43g755a9764a243a1be@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
	<abb391f80710251407k5e8c157blc80f3b1070e1aba1@mail.gmail.com>
	<fc6cbe440710251425v193f0d43g755a9764a243a1be@mail.gmail.com>
Message-ID: <fc6cbe440710251428g699b1365x7c566ce65055784c@mail.gmail.com>

Actually

http://blog.diginux.net/2007/04/03/writing-a-duplicate-file-finder-in-erlang/

would be a better link, it has the source attached

On 25/10/2007, Dale Harvey <harveyd@REDACTED> wrote:
>
> I actually read a recent article on planet erlang about locating duplicate
> files
> that has some information that might help
>
>
> http://planeterlang.org/story.php?title=Writing_a_duplicate_file_finder_in_Erlang
>
> On 25/10/2007, Linan Wang < tali.wang@REDACTED> wrote:
> >
> > Interesting problem.
> > It appears to me like a svn/cvs/git related problem. I think the core
> > is to merge indices. Have a look at git, in my memory it has such kind
> > of function.
> > Hopefully Time Machine will solve this kind of problems.
> >
> > On 25/10/2007, Joe Armstrong < erlang@REDACTED> wrote:
> > > I have an interesting? problem.
> > >
> > > Over the last ? years (> 10) I have been upgrading my home system
> > > this usually involved buying a bigger disk and copying most (or all)
> > > of the files
> > > from the old disk to the new disk or disks.
> > >
> > > I've also been backing up the family photos etc on USB disks.
> > >
> > > Now I have > 1 Tera bytes of files spread over c. 10 computers and
> > > 3 pluggable USB disks. Having made a "backup" both the original and
> > the
> > > copy live lives of their own.
> > >
> > > Does anybody know of a good algorithm to consolidate/merge all this
> > > data or do I have
> > > to write my own? One immediate thought is to compute the MD5 sums of
> > > all files on all
> > > disk and thus find all duplicates - then create a master copy of all
> > > unique files
> > > but the file names will be wrong and this might result in a big mess.
> > >
> > > This cannot be an uncommon problem - any ideas how to solve it?
> > >
> > > /Joe
> > > _______________________________________________
> > > erlang-questions mailing list
> > > erlang-questions@REDACTED
> > > http://www.erlang.org/mailman/listinfo/erlang-questions
> > >
> >
> >
> > --
> > Best regards
> >
> > Linan Wang
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
>
> --
> http://arandomurl.com/
> http://www.flickr.com/photos/daleharvey/




-- 
http://arandomurl.com/
http://www.flickr.com/photos/daleharvey/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/b7ddbd03/attachment.htm>

From yinso.chen@REDACTED  Thu Oct 25 23:39:47 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 14:39:47 -0700
Subject: [erlang-questions] disk merging
In-Reply-To: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
Message-ID: <779bf2730710251439u27ca6a07y7165ed88e91d59c5@mail.gmail.com>

Yeah you've described distributed version control problem ;)

If your repositories are basically different versions of the same thing -
i.e. you copy a part of the original tree out, and add/delete things but
didn't try to rename files), then Unison might be able to help you.  It's
designed to merge two sets of repository together based on file paths.  For
files with the same name it will attempt to detect which one is later, and
if it can't it will prompt you for reconciliation.

http://www.cis.upenn.edu/~bcpierce/unison/

If your problem is duplicate files with different names, then MD5/SHA1 will
help you find dupes across file names, but it can't solve the file
versioning problem.

W.r.t the folder structure issue, you can actually preserve the folder
structure if you convert the dupe files to symlinks (at least on non-windows
platform).

But if you have changing files with different names, then there probably
will be a manual effort involved if you want to version them as the same
file (either manually checkin to a source control, or ensure the files
following a naming convention and have a script checkin for you).

On 10/25/07, Joe Armstrong <erlang@REDACTED> wrote:
>
> I have an interesting? problem.
>
> Over the last ? years (> 10) I have been upgrading my home system
> this usually involved buying a bigger disk and copying most (or all)
> of the files
> from the old disk to the new disk or disks.
>
> I've also been backing up the family photos etc on USB disks.
>
> Now I have > 1 Tera bytes of files spread over c. 10 computers and
> 3 pluggable USB disks. Having made a "backup" both the original and the
> copy live lives of their own.
>
> Does anybody know of a good algorithm to consolidate/merge all this
> data or do I have
> to write my own? One immediate thought is to compute the MD5 sums of
> all files on all
> disk and thus find all duplicates - then create a master copy of all
> unique files
> but the file names will be wrong and this might result in a big mess.
>
> This cannot be an uncommon problem - any ideas how to solve it?
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/a2912c9e/attachment.htm>

From yinso.chen@REDACTED  Thu Oct 25 22:48:32 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 13:48:32 -0700
Subject: [erlang-questions] multiple instances of gen_server
Message-ID: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>

Hi all -

can there be multiple instances of a gen_server module on the same node?  I
couldn't find the answer googling (though I probably missed it), but reading
through the source code it seems that the server process is registered by
name, so by default it appears to be a singleton.

The gen_server start template from erlang mode and programming erlang looks
like

gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).

Could it be that to enable multiple instances, instead of using ?MODULE as
the name, I pass in a different name myself and explicitly pass the name
through the API to get to the correct server?

Any insights are appreciated, thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/24d8c23b/attachment.htm>

From mogorman@REDACTED  Thu Oct 25 23:58:00 2007
From: mogorman@REDACTED (Matthew O'Gorman)
Date: Thu, 25 Oct 2007 16:58:00 -0500
Subject: [erlang-questions] multiple instances of gen_server
In-Reply-To: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
References: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
Message-ID: <f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>

if you read the man page or the docs, you dont have to register a name
to start start_link, also you can pass any term for the name so you
can pass a pid for example or a tuple with a complex unique name.

anyways hope this helps

mog

On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> Hi all -
>
> can there be multiple instances of a gen_server module on the same node?  I
> couldn't find the answer googling (though I probably missed it), but reading
> through the source code it seems that the server process is registered by
> name, so by default it appears to be a singleton.
>
> The gen_server start template from erlang mode and programming erlang looks
> like
>
> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
>
> Could it be that to enable multiple instances, instead of using ?MODULE as
> the name, I pass in a different name myself and explicitly pass the name
> through the API to get to the correct server?
>
> Any insights are appreciated, thanks,
> yc
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From yinso.chen@REDACTED  Fri Oct 26 00:09:38 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 15:09:38 -0700
Subject: [erlang-questions] multiple instances of gen_server
In-Reply-To: <f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>
References: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
	<f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>
Message-ID: <779bf2730710251509v79906291ke64c3b1414699285@mail.gmail.com>

Thanks for the pointer.

Looks like to use an unnamed server it I need to capture the returned PID
and pass it through call (it wasn't clear that's the case via
gen_server.erl; but it's more clear w/in gen.erl), so that should work.

Thanks,
yc

On 10/25/07, Matthew O'Gorman <mogorman@REDACTED> wrote:
>
> if you read the man page or the docs, you dont have to register a name
> to start start_link, also you can pass any term for the name so you
> can pass a pid for example or a tuple with a complex unique name.
>
> anyways hope this helps
>
> mog
>
> On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> > Hi all -
> >
> > can there be multiple instances of a gen_server module on the same
> node?  I
> > couldn't find the answer googling (though I probably missed it), but
> reading
> > through the source code it seems that the server process is registered
> by
> > name, so by default it appears to be a singleton.
> >
> > The gen_server start template from erlang mode and programming erlang
> looks
> > like
> >
> > gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
> >
> > Could it be that to enable multiple instances, instead of using ?MODULE
> as
> > the name, I pass in a different name myself and explicitly pass the name
> > through the API to get to the correct server?
> >
> > Any insights are appreciated, thanks,
> > yc
> >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/bdedda94/attachment.htm>

From ulf@REDACTED  Fri Oct 26 01:10:17 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Fri, 26 Oct 2007 01:10:17 +0200
Subject: [erlang-questions] multiple instances of gen_server
In-Reply-To: <f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>
References: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
	<f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>
Message-ID: <8209f740710251610s10c85a21vbfccba0f7e39686f@mail.gmail.com>

2007/10/25, Matthew O'Gorman <mogorman@REDACTED>:
> if you read the man page or the docs, you dont have to register a name
> to start start_link, also you can pass any term for the name so you
> can pass a pid for example or a tuple with a complex unique name.

If you want to register a gen_server locally, the name must be an atom.
If you register it globally, the name can be any term.

If you don't register it at all, subsequent calls to the server instance must
use the pid.

BR,
Ulf W


From mogorman@REDACTED  Fri Oct 26 01:19:01 2007
From: mogorman@REDACTED (Matthew O'Gorman)
Date: Thu, 25 Oct 2007 18:19:01 -0500
Subject: [erlang-questions] multiple instances of gen_server
In-Reply-To: <8209f740710251610s10c85a21vbfccba0f7e39686f@mail.gmail.com>
References: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
	<f49fd40d0710251458u47577a79u22dc7bb2d9eb4fba@mail.gmail.com>
	<8209f740710251610s10c85a21vbfccba0f7e39686f@mail.gmail.com>
Message-ID: <f49fd40d0710251619k159476e1vbc52dcc7fa4ecc6b@mail.gmail.com>

you are correct, it would probably help if i read the docs more carefully....

mog

On 10/25/07, Ulf Wiger <ulf@REDACTED> wrote:
> 2007/10/25, Matthew O'Gorman <mogorman@REDACTED>:
> > if you read the man page or the docs, you dont have to register a name
> > to start start_link, also you can pass any term for the name so you
> > can pass a pid for example or a tuple with a complex unique name.
>
> If you want to register a gen_server locally, the name must be an atom.
> If you register it globally, the name can be any term.
>
> If you don't register it at all, subsequent calls to the server instance must
> use the pid.
>
> BR,
> Ulf W
>


From yinso.chen@REDACTED  Fri Oct 26 02:17:10 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 17:17:10 -0700
Subject: [erlang-questions] "dynamic" include?
Message-ID: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>

Hi all -

is it possible to include programmatically generated path?

i.e.

instead of

-include("../../foo.hrl").

write

-include(dynamic_determine("foo.hrl")).

I suspect not (the above don't compile for me), but the idea is that you
might not know for sure where the file would be, and rely on the vm to
figure it out via the code:which() mechanism.  For example, yaws can be
installed locally or globally, and if you need access to yaws hrl files, you
can't depend on it exist @ a particular location (assuming your code branch
has no relation with yaws' code branch), and it's a pain if one has to
modify the source if the locations differ.

What's the best practice to deal with such situation?  Any thoughts are
appreicated, thanks.
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/4fce5419/attachment.htm>

From rsaccon@REDACTED  Fri Oct 26 02:43:56 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 25 Oct 2007 21:43:56 -0300
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
Message-ID: <eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>

when you use CEAN and all your third party code is in CEAN, then you
have a bit less trouble with location of headerfiles.

You could also programmatically set your include directories in the
Makefile or use an Escript to compile, then you don't have to hardcode
those paths in your src files.


On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> Hi all -
>
> is it possible to include programmatically generated path?
>
> i.e.
>
> instead of
>
> -include("../../foo.hrl").
>
> write
>
> -include(dynamic_determine("foo.hrl")).
>
> I suspect not (the above don't compile for me), but the idea is that you
> might not know for sure where the file would be, and rely on the vm to
> figure it out via the code:which() mechanism.  For example, yaws can be
> installed locally or globally, and if you need access to yaws hrl files, you
> can't depend on it exist @ a particular location (assuming your code branch
> has no relation with yaws' code branch), and it's a pain if one has to
> modify the source if the locations differ.
>
> What's the best practice to deal with such situation?  Any thoughts are
> appreicated, thanks.
> yc
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
Roberto Saccon
http://rsaccon.com


From launoja@REDACTED  Fri Oct 26 00:20:31 2007
From: launoja@REDACTED (Jani Launonen)
Date: Fri, 26 Oct 2007 01:20:31 +0300
Subject: [erlang-questions] multiple instances of gen_server
In-Reply-To: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
References: <779bf2730710251348u773dd4c1r71add090c45b343b@mail.gmail.com>
Message-ID: <f51c9889bb02.472140df@omanetti.fi>



----- Alkuper?inen viesti -----
L?hett?j?: YC <yinso.chen@REDACTED>
P?iv?ys: perjantai, lokakuu 26, 2007 1:10 am
Aihe: [erlang-questions] multiple instances of gen_server
Vastaanottaja: erlang-questions Questions <erlang-questions@REDACTED>

> Hi all -
> 
> can there be multiple instances of a gen_server module on the 
> same node?? I
> couldn't find the answer googling (though I probably missed it), 
> but reading
> through the source code it seems that the server process is 
> registered by
> name, so by default it appears to be a singleton.
> 
> The gen_server start template from erlang mode and programming 
> erlang looks
> like
> 
> gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
> 
> Could it be that to enable multiple instances, instead of using 
> ?MODULE as
> the name, I pass in a different name myself and explicitly pass 
> the name
> through the API to get to the correct server?

The documentation says that (http://www.erlang.org/doc/design_principles/gen_server.html#2.3):

"start_link() ->
    gen_server:start_link({local, ch3}, ch3, [], []) => {ok, Pid}
    
start_link calls the function gen_server:start_link/4. This function spawns and links to a new process, a 
gen_server.


The first argument {local, ch3} specifies the name. In this case, the gen_server will be locally registered as 
ch3.
If the name is omitted, the gen_server is not registered. Instead its pid must be used. The name could also 
be given as {global, Name}, in which case the gen_server is registered using global:register_name/2."

So, yes. You could create unique names for each gen_server, or not to bother to name them at all. You could 
use only the PID to communicate with the gen_server instance. 
 
> Any insights are appreciated, thanks,
> yc

Cheers,
Jani L.


From yinso.chen@REDACTED  Fri Oct 26 03:49:52 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 18:49:52 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
Message-ID: <779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>

Hi Roberto -

On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:

> > You could also programmatically set your include directories in the
> > Makefile or use an Escript to compile, then you don't have to hardcode
> > those paths in your src files.


Are you talking about passing code_path into erl when writing Makefile?
It's not clear how that would enable the hrl to be dynamically set.  Do you
have any examples?

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/fe54c9f1/attachment.htm>

From rsaccon@REDACTED  Fri Oct 26 04:05:33 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Thu, 25 Oct 2007 23:05:33 -0300
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
Message-ID: <eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>

No, I was simply thinking about a shell script finding out whether for
example yaws is locally or globally installed. Me and many others had
this problem in the early days of erlyweb.

On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> Hi Roberto -
>
> On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> > > You could also programmatically set your include directories in the
> > > Makefile or use an Escript to compile, then you don't have to hardcode
> > > those paths in your src files.
>
> Are you talking about passing code_path into erl when writing Makefile?
> It's not clear how that would enable the hrl to be dynamically set.  Do you
> have any examples?
>  Thanks,
> yc
>
>


-- 
Roberto Saccon
http://rsaccon.com


From yinso.chen@REDACTED  Fri Oct 26 04:27:31 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 19:27:31 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
	<eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>
Message-ID: <779bf2730710251927j5b2f7201ycfb22c6d43908b2b@mail.gmail.com>

Thanks - is that shell script you referred to a part of the erlyweb disti?
Where can I find the shell script?

Thanks,
yc

On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
>
> No, I was simply thinking about a shell script finding out whether for
> example yaws is locally or globally installed. Me and many others had
> this problem in the early days of erlyweb.
>
> On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> > Hi Roberto -
> >
> > On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> > > > You could also programmatically set your include directories in the
> > > > Makefile or use an Escript to compile, then you don't have to
> hardcode
> > > > those paths in your src files.
> >
> > Are you talking about passing code_path into erl when writing Makefile?
> > It's not clear how that would enable the hrl to be dynamically set.  Do
> you
> > have any examples?
> >  Thanks,
> > yc
> >
> >
>
>
> --
> Roberto Saccon
> http://rsaccon.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/08620547/attachment.htm>

From rsaccon@REDACTED  Fri Oct 26 05:12:33 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Fri, 26 Oct 2007 00:12:33 -0300
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710251927j5b2f7201ycfb22c6d43908b2b@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
	<eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>
	<779bf2730710251927j5b2f7201ycfb22c6d43908b2b@mail.gmail.com>
Message-ID: <eb4177b00710252012r3bebfe86t87d9da24c7eaf5c9@mail.gmail.com>

On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> Thanks - is that shell script you referred to a part of the erlyweb disti?

not that I know ..

> Where can I find the shell script?

You either write your own glue (simple shellscript, automake based or
escript)  to connect your Erlang program with the Erlang VM, third
party libraries and your project specific environment around it or you
wait a few days for the launch of erlware.org repository, maybe that
will help to sort out these kind of problems.

>
> Thanks,
> yc
>
>
> On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> > No, I was simply thinking about a shell script finding out whether for
> > example yaws is locally or globally installed. Me and many others had
> > this problem in the early days of erlyweb.
> >
> > On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> > > Hi Roberto -
> > >
> > > On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> > > > > You could also programmatically set your include directories in the
> > > > > Makefile or use an Escript to compile, then you don't have to
> hardcode
> > > > > those paths in your src files.
> > >
> > > Are you talking about passing code_path into erl when writing Makefile?
> > > It's not clear how that would enable the hrl to be dynamically set.  Do
> you
> > > have any examples?
> > >  Thanks,
> > > yc
> > >
> > >
> >
> >
> > --
> > Roberto Saccon
> > http://rsaccon.com
> >
>
>


-- 
Roberto Saccon
http://rsaccon.com


From cyberlync@REDACTED  Fri Oct 26 05:36:42 2007
From: cyberlync@REDACTED (Eric Merritt)
Date: Thu, 25 Oct 2007 20:36:42 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
Message-ID: <f69c840c0710252036m37f70e2qe5d7ad390eea83be@mail.gmail.com>

Sinan sets up include paths correctly during build as well so you just
don't have to worry about where you includes are.


On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> when you use CEAN and all your third party code is in CEAN, then you
> have a bit less trouble with location of headerfiles.
>
> You could also programmatically set your include directories in the
> Makefile or use an Escript to compile, then you don't have to hardcode
> those paths in your src files.
>
>
> On 10/25/07, YC <yinso.chen@REDACTED> wrote:
> > Hi all -
> >
> > is it possible to include programmatically generated path?
> >
> > i.e.
> >
> > instead of
> >
> > -include("../../foo.hrl").
> >
> > write
> >
> > -include(dynamic_determine("foo.hrl")).
> >
> > I suspect not (the above don't compile for me), but the idea is that you
> > might not know for sure where the file would be, and rely on the vm to
> > figure it out via the code:which() mechanism.  For example, yaws can be
> > installed locally or globally, and if you need access to yaws hrl files,
> you
> > can't depend on it exist @ a particular location (assuming your code
> branch
> > has no relation with yaws' code branch), and it's a pain if one has to
> > modify the source if the locations differ.
> >
> > What's the best practice to deal with such situation?  Any thoughts are
> > appreicated, thanks.
> > yc
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> --
> Roberto Saccon
> http://rsaccon.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From jeffm@REDACTED  Fri Oct 26 07:28:50 2007
From: jeffm@REDACTED (jm)
Date: Fri, 26 Oct 2007 15:28:50 +1000
Subject: [erlang-questions] Using select in a Port
In-Reply-To: <200710252035.l9PKZkun046014@pluto.hedeland.org>
References: <200710252035.l9PKZkun046014@pluto.hedeland.org>
Message-ID: <47217B12.4070805@ghostgun.com>

Per Hedeland wrote:

> I can however be wrong from your process' point of view - e.g. if it
> expects to receive binaries in Erlang external term format, and
> optimistically does a binary_to_term/1 on whatever it gets, it will die
> with a badarg if your port program sends garbage - and the port will be
> closed *as a result of your process dying*. At least during debugging
> you may want to wrap catch and/or try around such things. And at least
> when running from the shell, you should get a message about the
> immediate cause of any "unnatural" death:
> 
> =ERROR REPORT==== 25-Oct-2007::22:30:11 ===
> Error in process <0.52.0> with exit value: {badarg,[{erlang,binary_to_term,[<<3 bytes>>]},{ll,loop,1}]}
> 
> 
> --Per

I put a simple try/catch around the loop/1 as you suggested

                 try loop(Port)
                   catch
                       T:Err -> io:format("~n**** catch : ~p:~p~n",
                                          [T, Err])
                   end

This did it. It spat out the error. A very dumb error, which when 
corrected rectified the problem. Given the amount of code I would have 
thought the error lay else where. I'd looked at that particular line of 
code a number of times and didn't see anything. Just goes to show. 
Anyway, I've added abit more code, fix a couple of other bugs, and 
removed all those fprintf() statements. The next chance I'll get to work 
on this after today won't be for another week or two at which time I'm 
hoping to be able to make an initial release.

Jeff.



From yinso.chen@REDACTED  Fri Oct 26 07:33:30 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 22:33:30 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <eb4177b00710252012r3bebfe86t87d9da24c7eaf5c9@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
	<eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>
	<779bf2730710251927j5b2f7201ycfb22c6d43908b2b@mail.gmail.com>
	<eb4177b00710252012r3bebfe86t87d9da24c7eaf5c9@mail.gmail.com>
Message-ID: <779bf2730710252233h61937f6bibfcdcc835be53c84@mail.gmail.com>

On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
>
> > > Where can I find the shell script?
>
> > You either write your own glue (simple shellscript, automake based or
> > escript)  to connect your Erlang program with the Erlang VM, third
> > party libraries and your project specific environment around it or you
> > wait a few days for the launch of erlware.org repository, maybe that
> > will help to sort out these kind of problems.


I see - just checking to see if there are reusable solutions.  I can check
on elware when it launches if I haven't cooked up something myself by then.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/117019ea/attachment.htm>

From yinso.chen@REDACTED  Fri Oct 26 07:36:01 2007
From: yinso.chen@REDACTED (YC)
Date: Thu, 25 Oct 2007 22:36:01 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <f69c840c0710252036m37f70e2qe5d7ad390eea83be@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<f69c840c0710252036m37f70e2qe5d7ad390eea83be@mail.gmail.com>
Message-ID: <779bf2730710252236p52412b7fwad3643ccea0b5793@mail.gmail.com>

On 10/25/07, Eric Merritt <cyberlync@REDACTED> wrote:
>
> > Sinan sets up include paths correctly during build as well so you just
> > don't have to worry about where you includes are.


Are you referring to http://code.google.com/p/sinan/?  It looks quite
interesting and I will check it out.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071025/187222a6/attachment.htm>

From cyberlync@REDACTED  Fri Oct 26 07:41:52 2007
From: cyberlync@REDACTED (Eric Merritt)
Date: Thu, 25 Oct 2007 22:41:52 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710252233h61937f6bibfcdcc835be53c84@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
	<eb4177b00710251743r73c49972n8be734abe4f171cb@mail.gmail.com>
	<779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
	<eb4177b00710251905q30fb5602y1acc8d6d29d7ab8d@mail.gmail.com>
	<779bf2730710251927j5b2f7201ycfb22c6d43908b2b@mail.gmail.com>
	<eb4177b00710252012r3bebfe86t87d9da24c7eaf5c9@mail.gmail.com>
	<779bf2730710252233h61937f6bibfcdcc835be53c84@mail.gmail.com>
Message-ID: <f69c840c0710252241k4dc5f5d6k4d16e30dd4f24a80@mail.gmail.com>

Sinan is under the Erlware umbrella. It should cover most of your
needs. We are shooting to launch on Nov. 1st so thats not so far away.

On 10/25/07, YC <yinso.chen@REDACTED> wrote:
>
> On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> > > > Where can I find the shell script?
> >
> > > You either write your own glue (simple shellscript, automake based or
> > > escript)  to connect your Erlang program with the Erlang VM, third
> > > party libraries and your project specific environment around it or you
> > > wait a few days for the launch of erlware.org repository, maybe that
> > > will help to sort out these kind of problems.
>
> I see - just checking to see if there are reusable solutions.  I can check
> on elware when it launches if I haven't cooked up something myself by then.
>
> Thanks,
> yc
>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From andreas.hillqvist@REDACTED  Fri Oct 26 08:45:08 2007
From: andreas.hillqvist@REDACTED (Andreas Hillqvist)
Date: Fri, 26 Oct 2007 08:45:08 +0200
Subject: [erlang-questions] Erlang FAQ out of date?
Message-ID: <8268eea30710252345h253fa833g78bb2452978f243a@mail.gmail.com>

I believe that the FAQ:
http://www.erlang.org/faq/faq.html#AEN196

Needs to be updated since the move to mailman.
Registrations should refer to:
http://www.erlang.org/mailman/listinfo

instead of:
"sending mail to  majordomo@REDACTED"


It was Eric/nindeu at TrapExit who noticed this. TrapExit refers to
the FAQ for registering to the mailing list.


Regards
Andreas Hillqvist


From tobbe@REDACTED  Fri Oct 26 08:45:43 2007
From: tobbe@REDACTED (Torbjorn Tornkvist)
Date: Fri, 26 Oct 2007 08:45:43 +0200
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
References: <779bf2730710251717n52426920nb53e95045ef37aab@mail.gmail.com>
Message-ID: <ffs2ep$lt0$1@ger.gmane.org>

YC wrote:
> Hi all -
> 
> is it possible to include programmatically generated path?
> 
> i.e.
> 
> instead of
> 
> -include("../../foo.hrl").
> 
> write
> 
> -include(dynamic_determine("foo.hrl")).

As it happends, I recently read (in the Erlang Handbook) that you
apparently should be able to do this:

 -include("$PATH/foo.hrl").

so in principle you should be able to do:

 -include("$WHATEVER").

I haven't verified that this works though...

Cheers, Tobbe


> 
> I suspect not (the above don't compile for me), but the idea is that you
> might not know for sure where the file would be, and rely on the vm to
> figure it out via the code:which() mechanism.  For example, yaws can be
> installed locally or globally, and if you need access to yaws hrl files,
> you can't depend on it exist @ a particular location (assuming your code
> branch has no relation with yaws' code branch), and it's a pain if one
> has to modify the source if the locations differ.
> 
> What's the best practice to deal with such situation?  Any thoughts are
> appreicated, thanks.
> yc
> 



From frej@REDACTED  Fri Oct 26 07:56:57 2007
From: frej@REDACTED (Frej Drejhammar)
Date: Fri, 26 Oct 2007 07:56:57 +0200
Subject: [erlang-questions] GUID
In-Reply-To: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
	(Joe Armstrong's message of "Thu, 25 Oct 2007 17:13:00 +0200")
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
Message-ID: <tz8r6ji4dw6.fsf@pingvin.sics.se>

> (or to read the MAC address of the local machine interfaces?)

On Linux:

cat /sys/class/net/eth0/address

--Frej


From yarivsadan@REDACTED  Fri Oct 26 09:12:45 2007
From: yarivsadan@REDACTED (Yariv Sadan)
Date: Fri, 26 Oct 2007 00:12:45 -0700
Subject: [erlang-questions] GUID
In-Reply-To: <eb4177b00710251025j559ab5dr7b1d5479397585c2@mail.gmail.com>
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
	<435AB503-84E5-48A1-AA9E-1C3686378EBE@uninet.ee>
	<eb4177b00710251025j559ab5dr7b1d5479397585c2@mail.gmail.com>
Message-ID: <17244f480710260012r3b609a88le286d370f15b5b8e@mail.gmail.com>

If you're going down the os:cmd() path, why not call 'os:cmd("uuidgen")'?

Yariv

On 10/25/07, Roberto Saccon <rsaccon@REDACTED> wrote:
> Something like this gives me the MAC address on OS X;
>
> os:cmd("ipconfig getpacket en0 | grep chaddr | awk '{print $3}'").
>
>
> On 10/25/07, Taavi Talvik <taavi@REDACTED> wrote:
> >
> > On Oct 25, 2007, at 6:13 PM, Joe Armstrong wrote:
> >
> > > Does anybody have some Erlang code to generate a GUID?
> > > (or to read the MAC address of the local machine interfaces?)
> >
> > Probably machine/os specific, on freebsd you can do
> >
> > 1> os:cmd("sysctl kern.hostuuid").
> > "kern.hostuuid: B5D5342C-9142-3AA0-81C8-7391B34506DA\n"
> >
> >
> > best regards,
> > taavi
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> --
> Roberto Saccon
> http://rsaccon.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From hans.bolinder@REDACTED  Fri Oct 26 10:44:53 2007
From: hans.bolinder@REDACTED (Hans Bolinder)
Date: Fri, 26 Oct 2007 10:44:53 +0200
Subject: [erlang-questions] Unexpected error report from Global name
	server
In-Reply-To: <70D00C33FCD1FD4A860DEAC228277C0C0309A2D7@MTNMAIL1.mtn.co.za>
References: <70D00C33FCD1FD4A860DEAC228277C0C0309A2D7@MTNMAIL1.mtn.co.za>
Message-ID: <18209.43269.222769.566901@gargle.gargle.HOWL>

[Trevor Woollacott:]
> I started 2 runtimes using -connect_all false
> 
> and get error reports when I connect the nodes.
...
> The global_name_server received an unexpected message:
> 
> handle_info({nodeup,admin3@REDACTED}, _)
...
> Is this an intended feature?

No, it isn't. Thanks for pointing it out. It will be fixed in R12B.

Best regards,

Hans Bolinder, Erlang/OTP team


From thomasl_erlang@REDACTED  Fri Oct 26 11:21:34 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Fri, 26 Oct 2007 02:21:34 -0700 (PDT)
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
Message-ID: <344532.31633.qm@web38815.mail.mud.yahoo.com>


--- YC <yinso.chen@REDACTED> wrote:

> Hi Roberto -
> 
> On 10/25/07, Roberto Saccon <rsaccon@REDACTED>
> wrote:
> 
> > > You could also programmatically set your include
> directories in the
> > > Makefile or use an Escript to compile, then you
> don't have to hardcode
> > > those paths in your src files.
> 
> 
> Are you talking about passing code_path into erl
> when writing Makefile?
> It's not clear how that would enable the hrl to be
> dynamically set.  Do you
> have any examples?

What would a "dynamically set" include path mean to
the compiler?

Anyway, the erlang compiler just uses the provided
include paths, much like a C compiler. To modify the
include path I tend to use things like:

   erlc -I$(INC) mymod.erl

or

   compile:file("mymod.erl", [{i, "..."}])

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From matthias@REDACTED  Fri Oct 26 13:11:19 2007
From: matthias@REDACTED (Matthias Lang)
Date: Fri, 26 Oct 2007 13:11:19 +0200
Subject: [erlang-questions] is anyone else using erlang docbuilder?
Message-ID: <18209.52055.446607.298180@cors.corelatus.se>

Hi,

I've been converting the FAQ source from 'docbook' to 'docbuilder' and
run into a few things with that "I must be missing something"
feeling. Maybe someone can point me in the right direction.

Problem #1: code/shell snippets in the document
-----------------------------------------

  With docbook, I could do this sort of thing for examples:

  <screen>
        1> self().
        <0.23.0>
        2> x - y.
        ** exited: {badarith,[{erl_eval,eval_op,3},
                      {erl_eval,exprs,4},
                      {shell,eval_loop,2}]} **
        3> self().
        <0.40.0>
  </screen>

  If I do the same in docbuilder, except with a <code> block, docbuilder
  chokes. Short of escaping every angle bracket, is there some way to
  do the above?

Problem #2: I can't put italic text in a title
----------------------------------------------

  Or: why doesn't the <em> tag seem to be allowed in a title?

Bug report: empty code blocks make docbuilder crash
---------------------------------------------------

   Writing
   
      <code></code>
   
   gets me
   
   39> docb_transform:file("top").
   Processing "getting_started.xml"
   *** Error: XML export error:
   {{badmatch,[]},
    [{docb_xmerl_tree_cb,'#element#',5},
     {xmerl,apply_cb,5},
     {xmerl,tagdef,5},
     {xmerl,export_content,2},
     {xmerl,export_content,2},
     {xmerl,export_content,2},
     {xmerl,export_element,2},
     {xmerl,export_content,2}]}
   *** Error: Parse error when building chapter how_do_i
   
   This is bad for several reasons. (a) I don't see why an empty code
   block is bad. (b) The crash talks about two different filenames, only
   one of which is the right one. (c) the crash doesn't tell me what line
   it crashed on, making it hard to find the problem in the first place.

Matthias



From matthias@REDACTED  Fri Oct 26 13:19:16 2007
From: matthias@REDACTED (Matthias Lang)
Date: Fri, 26 Oct 2007 13:19:16 +0200
Subject: [erlang-questions] GUID
In-Reply-To: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
References: <9b08084c0710250813j366fc2b2u48a2983d97e06512@mail.gmail.com>
Message-ID: <18209.52532.722650.646895@cors.corelatus.se>

Joe Armstrong writes:
 > Does anybody have some Erlang code to generate a GUID?
 > (or to read the MAC address of the local machine interfaces?)

>From some old code:

%%% There are some undocumented (but useful!) functions in inet:
%%%
%%%    inet:getif() returns all the IP addresses
%%%    inet:getiflist() returns all the network devices
%%%    inet:ifget("eth0", [hwaddr, addr]) returns the MAC and IP addrs

Matthias



From tonyg@REDACTED  Fri Oct 26 14:28:58 2007
From: tonyg@REDACTED (Tony Garnock-Jones)
Date: Fri, 26 Oct 2007 13:28:58 +0100
Subject: [erlang-questions] is anyone else using erlang docbuilder?
In-Reply-To: <18209.52055.446607.298180@cors.corelatus.se>
References: <18209.52055.446607.298180@cors.corelatus.se>
Message-ID: <4721DD8A.5020608@lshift.net>

Matthias Lang wrote:
>   With docbook, I could do this sort of thing for examples:
>   <screen>
>         <0.23.0>
>   </screen>

Was it SGML docbook? Is docbuilder XML?

Would a CDATA block make things easier?

<code><![CDATA[
  <0.23.0>
]]></code>

Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: tonyg@REDACTED


From per@REDACTED  Fri Oct 26 15:16:39 2007
From: per@REDACTED (Per Hedeland)
Date: Fri, 26 Oct 2007 15:16:39 +0200 (CEST)
Subject: [erlang-questions] GUID
In-Reply-To: <18209.52532.722650.646895@cors.corelatus.se>
Message-ID: <200710261316.l9QDGdZW066895@pluto.hedeland.org>

Matthias Lang <matthias@REDACTED> wrote:
>
>From some old code:
>
>%%% There are some undocumented (but useful!) functions in inet:
>%%%
>%%%    inet:getif() returns all the IP addresses
>%%%    inet:getiflist() returns all the network devices
>%%%    inet:ifget("eth0", [hwaddr, addr]) returns the MAC and IP addrs

You beat me to it.:-) Unfortunately the hwaddr part seems to work only
on Linux (tried FreeBSD and Solaris too) - which might be one reason
they're undocumented...

--Per Hedeland


From dcaoyuan@REDACTED  Fri Oct 26 16:51:42 2007
From: dcaoyuan@REDACTED (Caoyuan)
Date: Fri, 26 Oct 2007 22:51:42 +0800
Subject: [erlang-questions] widefinder update
In-Reply-To: <ca96bd270710250612x23c0b576maf9fe9c2e7dbf06c@mail.gmail.com>
References: <65b2728e0710231341n546e486dt5968832f40aa2d9a@mail.gmail.com>
	<417604.65860.qm@web38806.mail.mud.yahoo.com>
	<65b2728e0710240500k7a178721pf53aa591f766a633@mail.gmail.com>
	<ca96bd270710250612x23c0b576maf9fe9c2e7dbf06c@mail.gmail.com>
Message-ID: <ca96bd270710260751y7cc63a15qd565b11fe6a004a0@mail.gmail.com>

For my code, the best +h Size option is 8192, and the block size of
binary for processing + reading  is 10240000 to 20480000 Bytes per my
testing. The result is now about 2.97 sec vs Ruby's 4.1 sec on 2.8Ghz
4-CPU box.

Attached is the newest code with some cleanup. To evaluate:
$ erlc -smp tbray5.erl
$ time erl +h 8192 -smp -noshell -run tbray5 start o1000k.ap -s erlang halt

real    0m2.972s
user    0m9.685s
sys     0m0.748s


On 10/25/07, Caoyuan <dcaoyuan@REDACTED> wrote:
> It seems heap size is really a key for binary processing, and there
> are other tips for binary processing too. With proper heap size set,
> the straightforward Erlang code (in 80 LOC) can achieve around 3.1 sec
> on my 4-CPU linux box (the ruby code took about 4.1 sec on the same
> machine). The code is pasted on:
>
> http://blogtrader.net/page/dcaoyuan/entry/learning_coding_binary_was_tim
>
> With default heap size, the code may take 4.8+ sec.
>
> On 10/24/07, Steve Vinoski <vinoski@REDACTED> wrote:
> >
> > On 10/24/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
> > >
> > > --- Steve Vinoski <vinoski@REDACTED> wrote:
> > >
> > > Anders, thanks for collecting and posting these.
> > > > I've just performed a set
> > > > of new timings for all of them, as listed below. For
> > > > each, I just ran this
> > > > command:
> > > >
> > > > time erl -smp -noshell -run <test_case> main
> > > > o1000k.ap >/dev/null
> > > >
> > > > where "<test_case>" is the name of the tbray test
> > > > case file. All were
> > > > looped ten times, and I took the best timing for
> > > > each. All tests were done
> > > > on my 8-core 2.33 GHz dual Intel Xeon with 2 GB RAM
> > > > Linux box, in a local
> > > > (non-NFS) directory.
> > >
> > > So, looking at Steve's results on his 8-core system,
> > > we have:
> > >
> > >             real    user     tbray5/real    user/real
> > > tbray5      9.8     --       1.0            --
> > > tbray14     6.63    34.53    1.48            5.21
> > > tbray15     4.12    25.14    2.38           6.10
> > > tbray16     3.16    16.15    3.10           5.11
> > > tbray_tuple 2.28     8.61    4.30           3.78
> > > tbray_ets   1.87     7.42    5.24           3.97
> > > tbray_blkr  1.55     7.34    6.32           4.74
> > >
> > > tbray5/real is the speedup versus the baseline, while
> > > user/real is the speedup for each version due to
> > > parallelization.
> > >
> > > Thus, the latest version is 6.3 times faster than the
> > > first one. The parallel speedup is about the same in
> > > tbray5 and tbray_blkr, a very decent utilization of
> > > >50%, but the amount of work (user) has shrunk from
> > > (presumably more than) 34.53 seconds to 7.34 seconds.
> > >
> > > Tim Bray's original Erlang number on "his macbook"
> > > appears to be 34.16 seconds user (probably about the
> > > same real?). How does this compare to Ruby? Tim Bray
> > > reported that it needed 3.46 seconds real, again on
> > > his macbook. (As I understand it, all results here are
> > > for the big data set.)
> > >
> >
> > Yes, all results are for o1000k.ap, Tim's original large dataset. As for
> > Ruby, I just ran Tim's original code on the 8-core, and out of ten attempts
> > the best was:
> >
> >
> > real    0m2.210s
> > user    0m2.095s
> > sys     0m0.109s
> > --steve
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
>
>
> --
> - Caoyuan
>


-- 
- Caoyuan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tbray5.erl
Type: application/octet-stream
Size: 2663 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071026/74de589c/attachment.obj>

From lenartlad@REDACTED  Fri Oct 26 17:38:18 2007
From: lenartlad@REDACTED (Ladislav Lenart)
Date: Fri, 26 Oct 2007 17:38:18 +0200
Subject: [erlang-questions] Programming Erlang Exercise 8.11 revisited
In-Reply-To: <688f5b410710251034t35e9d681qe8a795caff9ed2fe@mail.gmail.com>
References: <688f5b410710251034t35e9d681qe8a795caff9ed2fe@mail.gmail.com>
Message-ID: <472209EA.6070308@volny.cz>

Hello,

answers are inlined.

Justin Giancola wrote:
> Hello,
> 
> Taking a look at the September mailing list
> (http://www.erlang.org/pipermail/erlang-questions/2007-September/029373.html),
> Charles asked about this problem, and although there were some
> proposed solutions, I'm afraid I don't quite understand.  First, to
> reiterate the problem,
> 
> "Write a function start(AnAtom, Fun) to register AnAtom as spawn(Fun).
> Make sure your program works correctly in the case when two parallel
> processes simultaneously evaluate start/2. In this case, you must
> guarantee that one of these processes succeeds and the other fails."

The proposed solution from the mail above:

start(AnAtom, Fun) ->
      case whereis(AnAtom) of
          undefined ->
              Pid = spawn(Fun),
              register(AnAtom, Pid);
          true ->
              true
      end.

has a race condition between calls whereis/1 and register/2.

> First off, I've read elsewhere that register/2 already prevents this
> kind of thing by maintaining a queue of registration requests that it
> processes in some definite order.  So you are guaranteed that two
> processes could never be registered with the same name, etc.  At the
> risk of sounding naive, doesn't the following function meet the
> problem's requirements?
> 
> start(AnAtom, Fun) ->
>     register(AnAtom, spawn(Fun)).

With this solution, when a call to register/2 fails, it is the process
that called it who gets killed and not the newly spawned process.

> Now, a potential problem here is that the phrasing of the problem is a
> little vague.  Does 'guarantee that one of these processes succeeds
> and the other fails' mean 'succeeds at registration', or 'succeeds at
> being spawned'?  If it's the former, is my proposed solution adequate?
>  If it's the latter, what exactly happens to the newly spawned process
> when
> 
>     register(AnAtom, spawn(Fun))
> 
> is called after AnAtom has already been registered?  Is the new
> process spawned and orphaned?  Does spawning the new process fail
> somehow?  Without knowing the answers to these questions, one could do
> something like:
> 
> start(AnAtom, Fun) ->
>     Pid = spawn(Fun),
>     try register(AnAtom, Pid)
>     catch
> 	error:_X -> {error,"atom already registered"}
>     after
> 	Pid ! die
>     end.

This works. However it requires that Fun will explicitly receive die
message and stop itself. It is also a problem if the spawned process
should not start doing anything (evaluating Fun) if it can not be
registered.

> to avoid encountering such issues.  However, this doesn't seem like a
> very concurrency-oriented solution.
> 
> In summary, I'm not sure I understand the intent of this problem.  As
> it's the first problem on concurrent programming in the book, I don't
> think it's intended to be that difficult.  On the other hand, neither
> of my approaches seems very useful.  Thoughts?

I posted the following code snippet as a reply to the original post
from Charles Gordon that seemed to solve the problem for him:

start(Atom, Fun) when is_atom(Atom), is_function(Fun, 0) ->
	Sender = self(),
	Fun2 = fun() ->
		case catch register(Atom, self()) of
			true ->
				Sender ! {started, self()},
				Fun();
			_ ->
				Sender ! {already_running, self()}
		end
	end,
	Pid = spawn(Fun2),
	receive
		{started, Pid} ->
			{ok, Pid};
		{already_running, Pid} ->
			already_running
	end.

Hope this helps,

Ladislav Lenart




From erlang@REDACTED  Fri Oct 26 19:56:16 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Fri, 26 Oct 2007 19:56:16 +0200
Subject: [erlang-questions] idea: tagging subject line in this group
Message-ID: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>

This group is getting pretty busy. It's also a useful resource for
finding out how to get this done.

Can I suggest a simple tagging of the subject line. As follows:

     idea:    An idea you want to share and get feedback on
     how:     A request for technical info

Replies are just follow ups to the original posting.

    erlounge: announce an erlounge
    beginner: a beginners question
    clarify:     please clarify ...
    program: I've written a fun program ...
    ad: an advertisement for something
    jobad: a job ad
    link: read this article it's interesting

/Joe


From erlang@REDACTED  Fri Oct 26 20:02:54 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Fri, 26 Oct 2007 20:02:54 +0200
Subject: [erlang-questions] how: can I get live keyboard input in the Erlang
	shell?
Message-ID: <9b08084c0710261102v2775f288rb598afc706dc4edc@mail.gmail.com>

There we go, I'm using my subject line tagging

Once upon a long time ago I knew how to do this (I wrote the code) -
but it was a long time ago
and the code has probably been rewritten many times. Is there a
defined way of doing this?

It would be nice to turn echoing off (say for inputting passwords) and
respond to *every* character.
While we're at it it would be nice to use curses commands to move the
cursor round the terminal.

Is this possible or do I have to rewrite the terminal IO ?

/Joe


From aaron.feng@REDACTED  Fri Oct 26 20:10:07 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Fri, 26 Oct 2007 14:10:07 -0400
Subject: [erlang-questions] idea: tagging subject line in this group
In-Reply-To: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>
References: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>
Message-ID: <4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>

I think this would be a great idea.  This will also make filtering out
messages much easier.

Aaron

On 10/26/07, Joe Armstrong <erlang@REDACTED> wrote:
>
> This group is getting pretty busy. It's also a useful resource for
> finding out how to get this done.
>
> Can I suggest a simple tagging of the subject line. As follows:
>
>      idea:    An idea you want to share and get feedback on
>      how:     A request for technical info
>
> Replies are just follow ups to the original posting.
>
>     erlounge: announce an erlounge
>     beginner: a beginners question
>     clarify:     please clarify ...
>     program: I've written a fun program ...
>     ad: an advertisement for something
>     jobad: a job ad
>     link: read this article it's interesting
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071026/55c361ac/attachment.htm>

From erlang@REDACTED  Fri Oct 26 20:26:17 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Fri, 26 Oct 2007 20:26:17 +0200
Subject: [erlang-questions] disk merging
In-Reply-To: <c5c75eb7dabc3e7ba259080648d28229@tombob.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
	<c5c75eb7dabc3e7ba259080648d28229@tombob.com>
Message-ID: <9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>

To answer my own question I think I have found  a good algorithm.

1) Start with an empty tree on machine 1 (the master tree)
 the goal is to move all files into the  master tree (or delete these files)

2) write code that adds directories to the master tree subject to a
number of rules

   a) The system is told where in the tree to add the files.

   or

  b) The system suggests places where the tree should be added. It
might offer several
       alternatives and the user selects which is correct.

   Nothing is added if any files in the new directory are already in
the tree (use md5 checksums to check this)

   If files in dir are very similar to the files in the master tree
(same name, small difference) then the
user is ask to choose which is "best" file.

Having moved files into the master tree on machine 1 I can replicate
the entire tree  on machine 2.


The key to this is a good algorithm to guess where in the master tree
new directories and individual files should be placed.

For images this is easy (I think) extract the date from the metadata
and move them
to /tree/pics/year/ ... here year comes from the image meta data.

Most of my images have names like  /some/funny/path/holidayInOozamba/xxx.jpg
If I know the year of xxx.jpg is 2001 (from the metadata) then I can
move these to
/tree/pics/2001/holidayInOozamba/xxx.jpg

I'd also use the md5 sum of xxx.jpg to make sure the file is not in
the master tree with
some different name.

Music? I need to guess the artist/album/track from a combination of
the filename and
any embedded Id3 tags. Now I know that about 5-10% of all mp3 files
have incorrect ID3 tags
(I found this writing the erlang book).

Are there any text data listings of (artict,album,track) tuples for
the 10^6 most popular
songs? - this would be useful.

If all else fails (I have a file xxx.mp3) with no decent tags, no file
or directory name hints,
is there some program that can predict the information I need? is
there a web service that can do this automatically?

/Joe






On 10/26/07, Robert Raschke <rrerlang@REDACTED> wrote:
>
> > Does anybody know of a good algorithm to consolidate/merge all this
> > data or do I have
> > to write my own? One immediate thought is to compute the MD5 sums of
> > all files on all
> > disk and thus find all duplicates - then create a master copy of all
> > unique files
> > but the file names will be wrong and this might result in a big mess.
> >
> > This cannot be an uncommon problem - any ideas how to solve it?
> >
> > /Joe
>
> For a low-level (i.e., file system) approach, have a look at Plan 9's Venti (consolidates on block level, not file):
>
> http://cm.bell-labs.com/sys/doc/venti/venti.html
>
> I believe there's an implementation that can run as a user level program under Unix in http://swtch.com/plan9port/
>
> Robby
>
>
>


From sgelkins@REDACTED  Fri Oct 26 22:26:30 2007
From: sgelkins@REDACTED (Steven Elkins)
Date: Fri, 26 Oct 2007 16:26:30 -0400
Subject: [erlang-questions] disk merging
In-Reply-To: <9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
	<c5c75eb7dabc3e7ba259080648d28229@tombob.com>
	<9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>
Message-ID: <ceb73c8f0710261326h4c0b459bkd3aa7cfc67cd25aa@mail.gmail.com>

On 10/26/07, Joe Armstrong <erlang@REDACTED> wrote:

>    Nothing is added if any files in the new directory are already in
> the tree (use md5 checksums to check this)

Use file size before computing checksums?


From yinso.chen@REDACTED  Fri Oct 26 23:46:33 2007
From: yinso.chen@REDACTED (YC)
Date: Fri, 26 Oct 2007 14:46:33 -0700
Subject: [erlang-questions] "dynamic" include?
In-Reply-To: <344532.31633.qm@web38815.mail.mud.yahoo.com>
References: <779bf2730710251849x502249b0wed1e458b5985f1d5@mail.gmail.com>
	<344532.31633.qm@web38815.mail.mud.yahoo.com>
Message-ID: <779bf2730710261446jee0f77avd3db9e9bb69e35ad@mail.gmail.com>

On 10/26/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
>
> > Anyway, the erlang compiler just uses the provided
> > include paths, much like a C compiler. To modify the
> > include path I tend to use things like:
>
> >    erlc -I$(INC) mymod.erl
>
> > or
>
> >   compile:file("mymod.erl", [{i, "..."}])


I see.  I got used to C-c C-k in erlang-mode already and forgot I can pass
in options.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071026/9a7a4f9e/attachment.htm>

From anders.nygren@REDACTED  Sat Oct 27 00:04:35 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 26 Oct 2007 17:04:35 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
Message-ID: <d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>

On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> To summarize my progress on the widefinder problem
> A few days ago I started with Steve Vinoski's tbray16.erl
> As a baseline on my 1.66 GHz dual core Centrino
> laptop, Linux,
> tbray16
> real    0m7.067s
> user    0m12.377s
> sys     0m0.584s
>
> I removed the dict used for the shift table,
> and changed the min_heap_size.
> That gave
> real    0m2.713s
> user    0m4.168s
> sys     0m0.412s
>
> (see tbray_tuple.erl and wfbm4_tuple.erl)
> Steve reported that it ran in ~1.9 s on his 8 core server.
>
> Then I removed the dicts that were used for collecting the
> matches and used ets instead, and got some improvement
> on my dual core laptop.
> real    0m2.220s
> user    0m3.252s
> sys     0m0.344s
>
> (see tbray_ets.erl and wfbm4_ets.erl)
>
> Interestingly Steve reported that it actually performed
> worse on his 8 core server.
>
> These versions all read the whole file into memory at the start.
> On my laptop that takes ~400ms (when the file is already cached
> in the OS).
>
> So I changed it to read the file in chucks and spawn the worker
> after each chunk is read.
>
> tbray_blockread with 4 processes
> real    0m1.992s
> user    0m3.176s
> sys     0m0.420s
>
> (see tbray_blockread.erl and wfbm4_ets.erl)
>
> Running it in the erlang shell it takes ~1.8s.
>
> Just starting and stopping the VM takes
> time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
>
> real    0m1.229s
> user    0m0.208s
> sys     0m0.020s
>
> It would be interesting to see how it runs on other machines,
> with more cores.
>
> /Anders
>
>

So I have a new version that I think will break the 1 second barrier
on Steve's 8-core
box.
The best I have seen on my dual core laptop is
real:  0m1.689s
user: 0m2.2756s
sys:  0m0.396s

The changes relative my latest posted tbray_blockread.erl are
- reading the file is in a separate process
- never bind variables to sub binaries unless absolutely necessary
- only have a limited number of worker processes at any time

One lesson from this exercise is that it can be bad for performance,
the result of changing the code to not bind variables to sub binaries
can be seen in the garbage collection statistics.

wfinder, (an unreleased version that ran in 1.050s on Steve's 8-core)
garbage collections: 46302
words reclaimed: 501768347

wfinder1
garbage collections: 13917
words reclaimed: 384561741

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfinder1.erl
Type: text/x-erlang
Size: 5796 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071026/d5c2397f/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_ets1.erl
Type: text/x-erlang
Size: 3512 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071026/d5c2397f/attachment-0001.bin>

From anders.nygren@REDACTED  Sat Oct 27 00:31:51 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Fri, 26 Oct 2007 17:31:51 -0500
Subject: [erlang-questions] R12 snapshots
Message-ID: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>

I have noticed that there are snapshots of R12 on erlang.org.

Is it any value to report bugs/problems on those, since there seems
to be a new snapshot every few hours.

I have tried a few during the last couple of days and hipe has been
broken in different ways.

/Anders


From patrickdlogan@REDACTED  Sat Oct 27 05:47:19 2007
From: patrickdlogan@REDACTED (Patrick Logan)
Date: Fri, 26 Oct 2007 20:47:19 -0700
Subject: [erlang-questions] GUID
In-Reply-To: <200710261316.l9QDGdZW066895@pluto.hedeland.org>
References: <18209.52532.722650.646895@cors.corelatus.se>
	<200710261316.l9QDGdZW066895@pluto.hedeland.org>
Message-ID: <e9447a400710262047l7e56a036q9b0e430dd3460b34@mail.gmail.com>

On linux the `uuid` command is more capable than the `uuidgen` command.

If yours has apt...

`sudo apt-get install uuid`

Or you can get it from...

http://www.ossp.org/


From jeffm@REDACTED  Sat Oct 27 06:01:37 2007
From: jeffm@REDACTED (jm)
Date: Sat, 27 Oct 2007 14:01:37 +1000
Subject: [erlang-questions] how: erlang interface to version control
Message-ID: <4722B821.5080003@ghostgun.com>

Is there a module floating around to interface erlang to any of the 
common revision control systems, eg subversion? Took a quick look in 
junerl and cean, but couldn't see anything.


Jeff.


From kostis@REDACTED  Sat Oct 27 07:52:18 2007
From: kostis@REDACTED (Kostis Sagonas)
Date: Sat, 27 Oct 2007 08:52:18 +0300
Subject: [erlang-questions] R12 snapshots
In-Reply-To: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
References: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
Message-ID: <4722D212.1030601@cs.ntua.gr>

Anders Nygren wrote:
 > I have noticed that there are snapshots of R12 on erlang.org.
 >
 > Is it any value to report bugs/problems on those, since there seems
 > to be a new snapshot every few hours.

Very little value in doing so.

 > I have tried a few during the last couple of days and hipe has been
 > broken in different ways.

\AFAIK, since Thursday evening, hipe should be working again.

Kostis


From casper2000a@REDACTED  Sat Oct 27 08:18:34 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Sat, 27 Oct 2007 11:48:34 +0530
Subject: [erlang-questions] Distrobuted/Fragmented Mnesia node crash
References: <fdtc5e$ec0$1@sea.gmane.org> <009401c804f1$d59b66e0$80d234a0$@com>
	<feg1d9$g7j$1@sea.gmane.org> 
Message-ID: <001201c81861$34256760$9c703620$@com>

I created a Fragmented Mnesia HowTo at
http://www.trapexit.org/Mnesia_Table_Fragmentation. Mnesia experts, pls let
me know if there's any wrong or better ways of doing things than how I have
written there.

BRgds,
- Eranga



-----Original Message-----
From: Eranga Udesh [mailto:casper2000a@REDACTED] 
Sent: Tuesday, October 09, 2007 10:31 PM
To: 'Torbjorn Tornkvist'; 'erlang-questions@REDACTED'
Subject: RE: [erlang-questions] Distrobuted/Fragmented Mnesia node crash

Well, the node suddenly terminates. I think getting Segmentation Fault. Node
is running as a Release and I used both ERL_CRASH_DUMP and
ERL_CRASH_DUMP_SECONDS env variables, but unfortunately the crash even
doesn't create the Crash Dump file. I will try to debug this further and
publish any findings I get. Meanwhile anybody else getting a similar issues,
pls share.

Yes, I will do a write up at trapexit about fragmented mnesia.

Brgds,
- Eranga




-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Torbjorn Tornkvist
Sent: Tuesday, October 09, 2007 7:28 PM
To: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Distrobuted/Fragmented Mnesia node crash

What errors do you get?

We know there is a lurking bug somewhere in the dets code.
We have got 'bad object' and 'premature eof' every other
month the last year. We have not been able to track the
bug down since the dets files is repaired automatically
next time it is opened.

As a temporary solution we restart mnesia whenever this error occurs...

Btw: it would be very nice if you had the time to write a little
trapexit-tutorial describing your fragmented setup.

Cheers, Tobbe

Eranga Udesh wrote:
> Hi,
> 
> I run 3 distributed Erlang nodes, dedicated to run Mnesia database server.
> Another Erlang node runs the application which read/write to those tables.
> Schema is created as disk_copies in all 4 nodes and tables are created as
> fragmented and distributed tables between 3 dedicated Mnesia nodes. I.e.
99
> fragments of a table distributed in 3 nodes resulting 33 fragments in
each.
> There're about 250-350 Db write/s and 500-800 DB read/s
> 
> Performance is quite good. I don't see "Mnesia overloaded" warnings.
However
> occasionally one of the Mnesia node crashes. Sometimes the time between 2
> crashes of the same Mnesia node may be about 1-2 days, while the next
crash
> takes about 10-20 days to happen. It doesn't even generate the crash_dump
> file or even if its generated, the size is 0 bytes.
> 
> Any idea what could be the cause? Are others also experience the same? I
> don't see any abnormal Load, Processor or Memory usage when these crashes
> occurs.
> 
> BRgds,
> - Eranga
> 
> 

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions




From rsaccon@REDACTED  Sat Oct 27 09:29:47 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Sat, 27 Oct 2007 04:29:47 -0300
Subject: [erlang-questions] anybody using textmate as Erlang editor ?
Message-ID: <eb4177b00710270029t2f838982ma40c5f4db058a028@mail.gmail.com>

this is a very nice mac text editor, with Erlang syntax highlighting.
Is anybody using this for Erlang coding and maybe even tried to  adapt
it a bit more to Erlang, a la Emacs (if that is possible at all) ?

regards
-- 
Roberto Saccon
http://rsaccon.com


From bjorn@REDACTED  Sat Oct 27 09:30:26 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 27 Oct 2007 09:30:26 +0200
Subject: [erlang-questions] R12 snapshots
In-Reply-To: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
References: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
Message-ID: <o11wbhggkt.fsf@erix.ericsson.se>

"Anders Nygren" <anders.nygren@REDACTED> writes:

> I have noticed that there are snapshots of R12 on erlang.org.
> 
> Is it any value to report bugs/problems on those, since there seems
> to be a new snapshot every few hours.

It is a bug in our update script. The snapshots are not really new
every hour, but the same snapshot with a new timestamp. I'll see if
I can fix that.

There will be new contents once every day.

We are only interested in reports about real bugs. :-)

The BEAM compiler and run-time system are supposed to be stable all the time,
so if something that worked in R11B doesn't work it is probably a bug.
But it could be a good idea to try at least two snapshots a few days apart
before reporting a potential bug.

> I have tried a few during the last couple of days and hipe has been
> broken in different ways.

Bugs in hipe are not worth reporting, unless the same bug occurs in R11B-5.

We (the Erlang/OTP group at Ericsson) often break hipe when we update
the compiler (e.g. adds new instructions) and/or run-time system.
The HiPE group in Uppsala mends hipe as soon as our code gets committed to
their CVS repository (which usually happens once a week).

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From sten@REDACTED  Sat Oct 27 09:54:38 2007
From: sten@REDACTED (Sten Kvamme)
Date: Sat, 27 Oct 2007 09:54:38 +0200
Subject: [erlang-questions] EX11
Message-ID: <AF5A563F-58C8-4EF1-A6C7-BB2E369BEF7F@kvamme.se>

Is there an ex11_lib.erl version 3.2 ? I have 3.1 but a 3.2 is  
mentioned in source code that I have seen...


By the way, ex11 works like a charm on Mac OS X, out of the box so to  
speak. http://www.kvamme.se/Aphasia/ex11_on_osx.gif

On Ubuntu there is an auth problem. I have tried xauth and xhost  
without success.

/Sten Kvamme


From ulf.wiger@REDACTED  Sat Oct 27 10:16:25 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Sat, 27 Oct 2007 10:16:25 +0200
Subject: [erlang-questions] how: erlang interface to version control
In-Reply-To: <4722B821.5080003@ghostgun.com>
References: <4722B821.5080003@ghostgun.com>
Message-ID: <4722F3D9.4030008@ericsson.com>

jm wrote:
> Is there a module floating around to interface erlang to any of the 
> common revision control systems, eg subversion? Took a quick look in 
> junerl and cean, but couldn't see anything.

I wrote a read-only CVS plugin to CCviewer once. It's not
in the version found on the erlang.org/contribs, but
you can get a tar file of the source code. Even though
you'd have to reverse-engineer the plugin interface, and
there is no documentation, you should figure it out
quickly enough. (:

CCviewer was a custom web server for source code browsing.
We used it for browsing Erlang code in Clearcase
repositories, but I thought it might be useful to support
other version control systems as well, so I made a plugin
interface to support different repositories, and wrote an
incomplete CVS plugin as a first experiment.

Since I had no need for updating CVS via the web server,
I haven't done any work on that.

(I've only attached the source code of the CVS plugin)

BR,
Ulf W
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ccviewer_cvs.tgz
Type: application/x-compressed
Size: 10149 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071027/82618eb6/attachment.bin>

From erlang@REDACTED  Sat Oct 27 13:37:00 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Sat, 27 Oct 2007 13:37:00 +0200
Subject: [erlang-questions] disk merging
In-Reply-To: <ceb73c8f0710261326h4c0b459bkd3aa7cfc67cd25aa@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
	<c5c75eb7dabc3e7ba259080648d28229@tombob.com>
	<9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>
	<ceb73c8f0710261326h4c0b459bkd3aa7cfc67cd25aa@mail.gmail.com>
Message-ID: <9b08084c0710270437n3d061b71n59c3b095832a5b0@mail.gmail.com>

Now sure - seems a premature optimization - also I need an extra data
base lookup
based on the file size.

/Joe

On 10/26/07, Steven Elkins <sgelkins@REDACTED> wrote:
> On 10/26/07, Joe Armstrong <erlang@REDACTED> wrote:
>
> >    Nothing is added if any files in the new directory are already in
> > the tree (use md5 checksums to check this)
>
> Use file size before computing checksums?
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From m94asr@REDACTED  Sat Oct 27 14:24:08 2007
From: m94asr@REDACTED (Armin Roehrl)
Date: Sat, 27 Oct 2007 14:24:08 +0200
Subject: [erlang-questions] jobad: Personifi, Texas, datamining,
	text-classification and high performance webservices
Message-ID: <c9841d290710270524k78444269k7cd43bf0245cf0cf@mail.gmail.com>

Freelance and permanent positions are available

* Fort Worth, TX,
* Telecommunicate is OK, so no green card needed.
* Small distributed international team working on leading edge AI-technologies
* erlang and ruby-skills are a must
* gentoo linux (we love gentoo)
* We want young and hungry developers that want to make history or
proven experienced gurus that enjoy doing something very challenging.
* You like algorithms, data-structures and parallelism :-)

If you are interested, please email a very short CV (pdf or txt, no word)
and most of all  reference to past projects to armin at personifi.com.

Thanks,
  -Armin Roehrl


From richardc@REDACTED  Sat Oct 27 16:50:22 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Sat, 27 Oct 2007 16:50:22 +0200
Subject: [erlang-questions] disk merging
In-Reply-To: <9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>
References: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>	<c5c75eb7dabc3e7ba259080648d28229@tombob.com>
	<9b08084c0710261126k3133f024hde51b7d900406e52@mail.gmail.com>
Message-ID: <4723502E.4090301@it.uu.se>

Joe Armstrong wrote:
> Music? I need to guess the artist/album/track from a combination of
> the filename and
> any embedded Id3 tags. Now I know that about 5-10% of all mp3 files
> have incorrect ID3 tags
> (I found this writing the erlang book).
> 
> Are there any text data listings of (artict,album,track) tuples for
> the 10^6 most popular
> songs? - this would be useful.

Better: use the MusicBrainz tagger ("Picard"): it usually works even
if the file is called xyz54321.mp3 and has no tags at all. It can
tag your files and give them sane names automatically.

    /Richard


From andre@REDACTED  Fri Oct 26 14:28:44 2007
From: andre@REDACTED (Andre Nathan)
Date: Fri, 26 Oct 2007 10:28:44 -0200
Subject: [erlang-questions] Erlang in a condor cluster
Message-ID: <1193401724.11697.9.camel@andre.mz.digirati.com.br>

Hello

Does anyone have experience running erlang using the condor
(http://www.cs.wisc.edu/condor/) cluster manager?

My application consists of a distributed graph (where each vertex is an
erlang process), and to use it I must submit jobs using condor. In the
job submition script,  I would have to start the erlang emulator on each
machine I am given, then from one machine start spawning processes, do
some computation on the graph and then shut down the emulators when the
computation is finished.

What would be the best way to acomplish that? I'm thinking of spawning
an extra process on each machine of the cluster that would call
init:stop() when receiving a 'terminate' message. Is there any better
approach for that?

Thanks in advance,
Andre



From jgrot@REDACTED  Sat Oct 27 18:25:17 2007
From: jgrot@REDACTED (Jonathan Grot)
Date: Sat, 27 Oct 2007 09:25:17 -0700 (PDT)
Subject: [erlang-questions] beginner: how: mnesia dir using slave:start()
Message-ID: <212378.79754.qm@web52709.mail.re2.yahoo.com>


Hello all,

I am trying to set the mnesia directory on a slave node in a way similar to the method suggested by the mnesia manual page:

Quoting the manual:
E = " -env DISPLAY " ++ net_adm:localhost() ++ ":0 ",
Arg = "-mnesia_dir " ++ M ++ " -pa " ++ Dir ++ E,
slave:start(H, Name, Arg).

The slave node gets launched as if I didn't specify a directory;  i.e. it creates a directory in the CWD.  I also tried a few alternatives with no success.

As a workaround, I wrote a shell script to launch the remote node using ssh.  This is okay, but I would like the option of using the slave module.  Any remedies?

Thanks!


Here is my test program:
=============================================
-module(test_mnesia2).
-export([run/1]).

run(Which)->
    % Report local mnesia directory
    LocalMdir = mnesia:system_info(directory),
    io:format("-----------------LOCAL MDIR--------------------~n~p~n",[LocalMdir]),

    % Choose launch method
    case Which of
    1 ->
        % Method suggested by mnesia manual
        slave:start(christina,b,"-mnesia_dir /tmp/MDB");
    2 ->
        % Try to launch like on command line
        slave:start(christina,b,"-mnesia dir /tmp/MDB");
    3 ->
        % Try again to launch like on command line
        slave:start(christina,b,"-mnesia dir \"/tmp/MDB\"")
    end,
    
    % Report remote mnesia directory
    RemoteMdir = rpc:call(b@REDACTED,mnesia,system_info,[directory]),
    io:format("-----------------REMOTE MDIR-------------------~n~p~n",[RemoteMdir]),

    % Kill remote node
    slave:stop(b@REDACTED),

    io:format("-----------------------------------------------~n").
=============================================

I launch the local shell with:
> erl -sname a -mnesia dir '"/tmp/MDB"'

And run my three cases:

(a@REDACTED)80> test_mnesia2:run(1).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
"/raid1/jgrot/erlang/Mnesia.b@REDACTED"
-----------------------------------------------
ok



(a@REDACTED)81> test_mnesia2:run(2).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
{badrpc,{'EXIT',{aborted,{aborted,{application_load_error,
                                      {bad_environment_value,"/tmp/MDB"}}}}}}

=ERROR REPORT==== 27-Oct-2007::12:05:22 ===
application_controller: syntax error before: '/': /tmp/MDB
-----------------------------------------------
ok



(a@REDACTED)82> test_mnesia2:run(3).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
{badrpc,{'EXIT',{aborted,{aborted,{application_load_error,
                                      {bad_environment_value,"/tmp/MDB"}}}}}}

=ERROR REPORT==== 27-Oct-2007::12:05:25 ===
application_controller: syntax error before: '/': /tmp/MDB
-----------------------------------------------
ok



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From aaron.feng@REDACTED  Sat Oct 27 20:05:31 2007
From: aaron.feng@REDACTED (Aaron Feng)
Date: Sat, 27 Oct 2007 14:05:31 -0400
Subject: [erlang-questions] Hierarchy of gen_server
Message-ID: <4e3934860710271105j403d28b2me42e35c02e85d929@mail.gmail.com>

Is hierarchy of gen_servers a preferred way to design an erlang
application?  Or this should be avoided and use a more parallel gen_server
relationship?

In either case, what is the best way for gen_servers communicate with each
other?  Should the pid be passed into the gen_server that needs to
communicate the other gen_server, or just use gen_server's registered name?

Aaron
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071027/b30f034d/attachment.htm>

From jgrot@REDACTED  Sat Oct 27 20:52:13 2007
From: jgrot@REDACTED (Jonathan Grot)
Date: Sat, 27 Oct 2007 11:52:13 -0700 (PDT)
Subject: [erlang-questions] beginner: how: mnesia dir using slave:start()
Message-ID: <134164.64917.qm@web52710.mail.re2.yahoo.com>

Minor correction: the manual page I am quoting is the slave manual, not the mnesia manual.
-Jon

----- Original Message ----
From: Jonathan Grot <jgrot@REDACTED>
To: erlang-questions@REDACTED
Sent: Saturday, October 27, 2007 12:25:17 PM
Subject: [erlang-questions] beginner: how: mnesia dir using slave:start()



Hello all,

I am trying to set the mnesia directory on a slave node in a way
 similar to the method suggested by the mnesia manual page:

Quoting the manual:
E = " -env DISPLAY " ++ net_adm:localhost() ++ ":0 ",
Arg = "-mnesia_dir " ++ M ++ " -pa " ++ Dir ++ E,
slave:start(H, Name, Arg).

The slave node gets launched as if I didn't specify a directory;  i.e.
 it creates a directory in the CWD.  I also tried a few alternatives
 with no success.

As a workaround, I wrote a shell script to launch the remote node using
 ssh.  This is okay, but I would like the option of using the slave
 module.  Any remedies?

Thanks!


Here is my test program:
=============================================
-module(test_mnesia2).
-export([run/1]).

run(Which)->
    % Report local mnesia directory
    LocalMdir = mnesia:system_info(directory),
    io:format("-----------------LOCAL
 MDIR--------------------~n~p~n",[LocalMdir]),

    % Choose launch method
    case Which of
    1 ->
        % Method suggested by mnesia manual
        slave:start(christina,b,"-mnesia_dir /tmp/MDB");
    2 ->
        % Try to launch like on command line
        slave:start(christina,b,"-mnesia dir /tmp/MDB");
    3 ->
        % Try again to launch like on command line
        slave:start(christina,b,"-mnesia dir \"/tmp/MDB\"")
    end,
    
    % Report remote mnesia directory
    RemoteMdir = rpc:call(b@REDACTED,mnesia,system_info,[directory]),
    io:format("-----------------REMOTE
 MDIR-------------------~n~p~n",[RemoteMdir]),

    % Kill remote node
    slave:stop(b@REDACTED),

    io:format("-----------------------------------------------~n").
=============================================

I launch the local shell with:
> erl -sname a -mnesia dir '"/tmp/MDB"'

And run my three cases:

(a@REDACTED)80> test_mnesia2:run(1).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
"/raid1/jgrot/erlang/Mnesia.b@REDACTED"
-----------------------------------------------
ok



(a@REDACTED)81> test_mnesia2:run(2).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
{badrpc,{'EXIT',{aborted,{aborted,{application_load_error,
                                    
  {bad_environment_value,"/tmp/MDB"}}}}}}

=ERROR REPORT==== 27-Oct-2007::12:05:22 ===
application_controller: syntax error before: '/': /tmp/MDB
-----------------------------------------------
ok



(a@REDACTED)82> test_mnesia2:run(3).
-----------------LOCAL MDIR--------------------
"/tmp/MDB"
-----------------REMOTE MDIR-------------------
{badrpc,{'EXIT',{aborted,{aborted,{application_load_error,
                                    
  {bad_environment_value,"/tmp/MDB"}}}}}}

=ERROR REPORT==== 27-Oct-2007::12:05:25 ===
application_controller: syntax error before: '/': /tmp/MDB
-----------------------------------------------
ok



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions




__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From vychodil.hynek@REDACTED  Sun Oct 28 08:21:24 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 08:21:24 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
Message-ID: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>

Hello,
These results are interesting, but I demur to kind of solution. Your
and Steve's approach have some caveats.

1/ File is read all in memory. When workers are so much slow, it can
happen principally. 200MB of Tim Bray's data is not problem on your
8CPU box, but what if file will be bigger. What about 1GB? No problem?
And 1TB? Still no problem? I know, that current i/o HW (and you don't
flush caches between measures and workers on 8CPU box are still fast
enough) can't provide data in performance causing problem for this
simple Tim Bray's exercise, but it is principally problem.

2/ Workers share resource (ets table) and it is principally bad. If
you have more CPU consuming task and you must use more CPU than as
current task to consume your input data bandwitch and  simultaneously
more result extensive task, you fall in trouble again.

As conclusion I think, your solution scale bad for both end. When you
have small amount of CPUs, you run out memory on larger datasets. When
you have more CPU, you fall in bottle neck of your shared resource. Of
course, Tim Bray's exercise is more CPU consuming than result
extensive and you don't fall to bottle neck trap and file reading on
current HW must be sequential and i/o performance is so bad, thus 8
CPU is enough to consume data faster than i/o can produce and you
don't run out of memory. But I think Tim Bray's exercise is not about
tuning solution for this one task, I think Tim Bray's exercise is
about multicore crisis and principal solutions.

Cheers,

--Hynek (Pichi) Vychodil

On 10/26/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > To summarize my progress on the widefinder problem
> > A few days ago I started with Steve Vinoski's tbray16.erl
> > As a baseline on my 1.66 GHz dual core Centrino
> > laptop, Linux,
> > tbray16
> > real    0m7.067s
> > user    0m12.377s
> > sys     0m0.584s
> >
> > I removed the dict used for the shift table,
> > and changed the min_heap_size.
> > That gave
> > real    0m2.713s
> > user    0m4.168s
> > sys     0m0.412s
> >
> > (see tbray_tuple.erl and wfbm4_tuple.erl)
> > Steve reported that it ran in ~1.9 s on his 8 core server.
> >
> > Then I removed the dicts that were used for collecting the
> > matches and used ets instead, and got some improvement
> > on my dual core laptop.
> > real    0m2.220s
> > user    0m3.252s
> > sys     0m0.344s
> >
> > (see tbray_ets.erl and wfbm4_ets.erl)
> >
> > Interestingly Steve reported that it actually performed
> > worse on his 8 core server.
> >
> > These versions all read the whole file into memory at the start.
> > On my laptop that takes ~400ms (when the file is already cached
> > in the OS).
> >
> > So I changed it to read the file in chucks and spawn the worker
> > after each chunk is read.
> >
> > tbray_blockread with 4 processes
> > real    0m1.992s
> > user    0m3.176s
> > sys     0m0.420s
> >
> > (see tbray_blockread.erl and wfbm4_ets.erl)
> >
> > Running it in the erlang shell it takes ~1.8s.
> >
> > Just starting and stopping the VM takes
> > time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
> >
> > real    0m1.229s
> > user    0m0.208s
> > sys     0m0.020s
> >
> > It would be interesting to see how it runs on other machines,
> > with more cores.
> >
> > /Anders
> >
> >
>
> So I have a new version that I think will break the 1 second barrier
> on Steve's 8-core
> box.
> The best I have seen on my dual core laptop is
> real:  0m1.689s
> user: 0m2.2756s
> sys:  0m0.396s
>
> The changes relative my latest posted tbray_blockread.erl are
> - reading the file is in a separate process
> - never bind variables to sub binaries unless absolutely necessary
> - only have a limited number of worker processes at any time
>
> One lesson from this exercise is that it can be bad for performance,
> the result of changing the code to not bind variables to sub binaries
> can be seen in the garbage collection statistics.
>
> wfinder, (an unreleased version that ran in 1.050s on Steve's 8-core)
> garbage collections: 46302
> words reclaimed: 501768347
>
> wfinder1
> garbage collections: 13917
> words reclaimed: 384561741
>
> /Anders
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>


From rrerlang@REDACTED  Fri Oct 26 11:57:46 2007
From: rrerlang@REDACTED (Robert Raschke)
Date: Fri, 26 Oct 2007 10:57:46 +0100
Subject: [erlang-questions] disk merging
In-Reply-To: <9b08084c0710251346y174a849gfbfa2eefeab61e62@mail.gmail.com>
Message-ID: <8794e43844bd99293e8b1b8a80fd8ac1@tombob.com>


> Does anybody know of a good algorithm to consolidate/merge all this
> data or do I have
> to write my own? One immediate thought is to compute the MD5 sums of
> all files on all
> disk and thus find all duplicates - then create a master copy of all
> unique files
> but the file names will be wrong and this might result in a big mess.
> 
> This cannot be an uncommon problem - any ideas how to solve it?
> 
> /Joe

For a low-level (i.e., file system) approach, have a look at Plan 9's Venti (consolidates on block level, not file):

http://cm.bell-labs.com/sys/doc/venti/venti.html

I believe there's an implementation that can run as a user level program under Unix in http://swtch.com/plan9port/

Robby




From thomasl_erlang@REDACTED  Sun Oct 28 13:29:51 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Sun, 28 Oct 2007 05:29:51 -0700 (PDT)
Subject: [erlang-questions] widefinder update
In-Reply-To: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
Message-ID: <992686.15039.qm@web38809.mail.mud.yahoo.com>


--- Hynek Vychodil <vychodil.hynek@REDACTED> wrote:

> Hello,
> These results are interesting, but I demur to kind
> of solution. Your
> and Steve's approach have some caveats.
> 
> 1/ File is read all in memory. 

Hynek,

This is true for some versions, but not all. The
'block read' version reads the file in chunks.

> 2/ Workers share resource (ets table) and it is
> principally bad. If
> you have more CPU consuming task and you must use
> more CPU than as
> current task to consume your input data bandwitch
> and  simultaneously
> more result extensive task, you fall in trouble
> again.

Note that the ets table in all proposals but one is
managed by a single process. It is just used as a more
efficient data structure. So the potential problem
here is really if this process becomes a bottleneck.

So, we have so far looked at two extremes:

1. Every worker maintains a local count, these are
then merged into a global count.

2. A single process maintains the global count,
workers send it updates.

But if this becomes problematic, one could also
combine the two by having 1 to N centralized counting
processes to trade off the cost of merging versus the
cost of incrementally sending all counts to a
'master'. (And one could batch the sending of updates
too, come to think of it.)

> As conclusion I think, your solution scale bad for
> both end. When you
> have small amount of CPUs, you run out memory on
> larger datasets. 

Not necessarily. With the block read solution, it
doesn't seem like you run that risk. 

The use of file:read_file/1 just showed that you
_could_ do fast I/O in Erlang, at a time when people
thought Erlang file I/O was very slow indeed. Showing
this was done by switching to a more suitable API
call. But you can be even more sophisticated than
that, e.g., by using file:pread.

> When
> you have more CPU, you fall in bottle neck of your
> shared resource. 

Do you mean that the problem becomes I/O bound? Do
note that all sufficiently fast solutions will
ultimately be limited by a hardware bottleneck of some
sort: CPU, I/O, network ...

In this particular case, you could increase I/O
performance by, say, striping the disk. And you can
increase CPU performance by, say, distributing the
work to multiple hosts/nodes (fairly straightforward
with Erlang, by the way). But with these problems,
even with infinite hardware you will eventually run
into some sequential portion of the code, and that
will limit the speedup as per Amdahl's Law.

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From vychodil.hynek@REDACTED  Sun Oct 28 14:06:11 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 14:06:11 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <992686.15039.qm@web38809.mail.mud.yahoo.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
Message-ID: <4d08db370710280606r1c8dd787i293e989f3fcceb5c@mail.gmail.com>

On 10/28/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
>
> > Hello,
> > These results are interesting, but I demur to kind
> > of solution. Your
> > and Steve's approach have some caveats.
> >
> > 1/ File is read all in memory.
>
> Hynek,
>
> This is true for some versions, but not all. The
> 'block read' version reads the file in chunks.

What version do you mean? tbray_blockread.erl from
http://www.erlang.org/pipermail/erlang-questions/2007-October/030118.html
reads in chunks, but when workers are slow you run out of memory. Look
at scan_file/9 cycle. There isn't limit of blocks in memory.

>
> > 2/ Workers share resource (ets table) and it is
> > principally bad. If
> > you have more CPU consuming task and you must use
> > more CPU than as
> > current task to consume your input data bandwitch
> > and  simultaneously
> > more result extensive task, you fall in trouble
> > again.
>
> Note that the ets table in all proposals but one is
> managed by a single process. It is just used as a more
> efficient data structure. So the potential problem
> here is really if this process becomes a bottleneck.
>
> So, we have so far looked at two extremes:
>
> 1. Every worker maintains a local count, these are
> then merged into a global count.
>
> 2. A single process maintains the global count,
> workers send it updates.
>
> But if this becomes problematic, one could also
> combine the two by having 1 to N centralized counting
> processes to trade off the cost of merging versus the
> cost of incrementally sending all counts to a
> 'master'. (And one could batch the sending of updates
> too, come to think of it.)
>
> > As conclusion I think, your solution scale bad for
> > both end. When you
> > have small amount of CPUs, you run out memory on
> > larger datasets.
>
> Not necessarily. With the block read solution, it
> doesn't seem like you run that risk.
>

Yes, but where is this solution? I can't see it in this thread now.
May be missed some, but solutions what I read are reader depend and
reader is not waiting for workers.

>
> The use of file:read_file/1 just showed that you
> _could_ do fast I/O in Erlang, at a time when people
> thought Erlang file I/O was very slow indeed. Showing
> this was done by switching to a more suitable API
> call. But you can be even more sophisticated than
> that, e.g., by using file:pread.
>
> > When
> > you have more CPU, you fall in bottle neck of your
> > shared resource.
>
> Do you mean that the problem becomes I/O bound? Do
> note that all sufficiently fast solutions will
> ultimately be limited by a hardware bottleneck of some
> sort: CPU, I/O, network ...
>
> In this particular case, you could increase I/O
> performance by, say, striping the disk. And you can
> increase CPU performance by, say, distributing the
> work to multiple hosts/nodes (fairly straightforward
> with Erlang, by the way). But with these problems,
> even with infinite hardware you will eventually run
> into some sequential portion of the code, and that
> will limit the speedup as per Amdahl's Law.
>

Yes, you are true. There isn't "best" solution. But at least make
memory safe solution we can.

Cheers
-- Hynek (Pichi) Vychodil


From anders.nygren@REDACTED  Sun Oct 28 14:12:14 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Sun, 28 Oct 2007 08:12:14 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <992686.15039.qm@web38809.mail.mud.yahoo.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
Message-ID: <d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>

On 10/28/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
>
> > Hello,
> > These results are interesting, but I demur to kind
> > of solution. Your
> > and Steve's approach have some caveats.
> >
> > 1/ File is read all in memory.
>
> Hynek,
>
> This is true for some versions, but not all. The
> 'block read' version reads the file in chunks.

It is still "sort of" true for the blockread and later versions,
since there is no flowcontrol, so when the file is already
cached in the OS the reading is faster than the processing and
all (almost) of the file will be in memory.
I am aware of this but have not bother with adding the flow
control yet.

>
> > 2/ Workers share resource (ets table) and it is
> > principally bad. If
> > you have more CPU consuming task and you must use
> > more CPU than as
> > current task to consume your input data bandwitch
> > and  simultaneously
> > more result extensive task, you fall in trouble
> > again.
>
> Note that the ets table in all proposals but one is
> managed by a single process. It is just used as a more
> efficient data structure. So the potential problem
> here is really if this process becomes a bottleneck.
>
> So, we have so far looked at two extremes:
>
> 1. Every worker maintains a local count, these are
> then merged into a global count.
>
> 2. A single process maintains the global count,
> workers send it updates.
>
> But if this becomes problematic, one could also
> combine the two by having 1 to N centralized counting
> processes to trade off the cost of merging versus the
> cost of incrementally sending all counts to a
> 'master'. (And one could batch the sending of updates
> too, come to think of it.)
>

I have not seen this as a problem yet since there is a relative
small number of concurrent workers. However as the number of
cores grow it may become a problem.
An alternative is that each worker has a ets tables for its counters and
sends its results to the central ets table on termination.

> > As conclusion I think, your solution scale bad for
> > both end. When you
> > have small amount of CPUs, you run out memory on
> > larger datasets.
>
> Not necessarily. With the block read solution, it
> doesn't seem like you run that risk.

See above.

>
> The use of file:read_file/1 just showed that you
> _could_ do fast I/O in Erlang, at a time when people
> thought Erlang file I/O was very slow indeed. Showing
> this was done by switching to a more suitable API
> call. But you can be even more sophisticated than
> that, e.g., by using file:pread.
>
> > When
> > you have more CPU, you fall in bottle neck of your
> > shared resource.
>
> Do you mean that the problem becomes I/O bound? Do
> note that all sufficiently fast solutions will
> ultimately be limited by a hardware bottleneck of some
> sort: CPU, I/O, network ...
>
> In this particular case, you could increase I/O
> performance by, say, striping the disk. And you can
> increase CPU performance by, say, distributing the
> work to multiple hosts/nodes (fairly straightforward
> with Erlang, by the way). But with these problems,
> even with infinite hardware you will eventually run
> into some sequential portion of the code, and that
> will limit the speedup as per Amdahl's Law.
>

Currently that sequential part is ~ 0.5s on my 1.66GHz
dual core laptop.
the part of the work that can be run in parallel takes
~2.254 s
so theoretically we would get
Cores  Real time  Speedup  Rel. speedup by doubling #cores
1         2.754	
2         1.627       1.693      1.693
4         1.064	      2.590      1.530
8         0.782	      3.523      1.360
16       0.641	     4.297      1.220
32       0.570	     4.828      1.123
64       0.535	     5.146      1.066
128     0.518	    5.321      1.034
256     0.509	    5.413      1.017

Which is not very good after 8 cores.

So I am now looking at making this a 'real' distributed solution instead.

/Anders


From vychodil.hynek@REDACTED  Sun Oct 28 15:18:28 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 15:18:28 +0100
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
Message-ID: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>

Hello,
I don't know how to write specification of function with many
patterns. For example how to specify this simple function

foo(X, Y) when is_integer(X), is_integer(Y) -> true;
foo(X, Y) when is_atom(X), is_atom(Y) -> false.

What should I write when I want to see in documentation that there
must be both parameters same type. I would like see in documentation
some like

foo(X::integer(), Y::integer()) -> true;
foo(X::atom(), Y::atom()) -> false.

Best
-- Hynek (Pichi) Vychodil


From vychodil.hynek@REDACTED  Sun Oct 28 15:32:28 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 15:32:28 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
Message-ID: <4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>

>
> Currently that sequential part is ~ 0.5s on my 1.66GHz
> dual core laptop.
> the part of the work that can be run in parallel takes
> ~2.254 s
> so theoretically we would get
> Cores  Real time  Speedup  Rel. speedup by doubling #cores
> 1         2.754
> 2         1.627       1.693      1.693
> 4         1.064       2.590      1.530
> 8         0.782       3.523      1.360
> 16       0.641       4.297      1.220
> 32       0.570       4.828      1.123
> 64       0.535       5.146      1.066
> 128     0.518       5.321      1.034
> 256     0.509       5.413      1.017
>
> Which is not very good after 8 cores.
>
> So I am now looking at making this a 'real' distributed solution instead.
>
> /Anders

I wrote some code to try parallel splitting, but it is still with
single dispatcher, because you must have single reader. Sequential
reading is about twenty times faster than random and you must keep
information about block order. Let you see chunk_reader and
nlt_reader.

Crazy think that chunk_reader is slower when compiled with native
option. I don't know why.

-- Hynek (Pichi) Vychodil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: chunk_reader.erl
Type: application/octet-stream
Size: 3722 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nlt_reader.erl
Type: application/octet-stream
Size: 7985 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment-0001.obj>

From vychodil.hynek@REDACTED  Sun Oct 28 16:21:57 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 16:21:57 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
Message-ID: <4d08db370710280821x47d39dep6a57fcf149549f5c@mail.gmail.com>

>
> I wrote some code to try parallel splitting, but it is still with
> single dispatcher, because you must have single reader. Sequential
> reading is about twenty times faster than random and you must keep
> information about block order. Let you see chunk_reader and
> nlt_reader.
>
> Crazy think that chunk_reader is slower when compiled with native
> option. I don't know why.
>
> -- Hynek (Pichi) Vychodil
>
>
And complete with this file_map_reduce.erl and wf_pichi1.erl code.
In wf_pichi1.erl is some my very old code but there is also much
better Anders finder from wfbm4_ets1 and works well. This is code with
one ets which can have bottle neck with many CPUs, of course. I know
it.

-- Hynek (Pichi) Vychodil
-------------- next part --------------
A non-text attachment was scrubbed...
Name: file_map_reduce.erl
Type: application/octet-stream
Size: 1931 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/207cb882/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wf_pichi1.erl
Type: application/octet-stream
Size: 5599 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/207cb882/attachment-0001.obj>

From thomasl_erlang@REDACTED  Sun Oct 28 17:51:15 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Sun, 28 Oct 2007 09:51:15 -0700 (PDT)
Subject: [erlang-questions] widefinder update
In-Reply-To: <4d08db370710280606r1c8dd787i293e989f3fcceb5c@mail.gmail.com>
Message-ID: <680592.8796.qm@web38810.mail.mud.yahoo.com>


--- Hynek Vychodil <vychodil.hynek@REDACTED> wrote:

> On 10/28/07, Thomas Lindgren
> <thomasl_erlang@REDACTED> wrote:
> >
> > --- Hynek Vychodil <vychodil.hynek@REDACTED>
> wrote:
> >
> > > Hello,
> > > These results are interesting, but I demur to
> kind
> > > of solution. Your
> > > and Steve's approach have some caveats.
> > >
> > > 1/ File is read all in memory.
> >
> > Hynek,
> >
> > This is true for some versions, but not all. The
> > 'block read' version reads the file in chunks.
> 
> What version do you mean? tbray_blockread.erl from
>
http://www.erlang.org/pipermail/erlang-questions/2007-October/030118.html
> reads in chunks, but when workers are slow you run
> out of memory. Look
> at scan_file/9 cycle. There isn't limit of blocks in
> memory.

Correct, the code doesn't support it at the moment,
but neither is there an inherent problem with adding
such a limit. If you are processing terabytes of data,
this is obviously a more pressing issue than at the
moment, but you are not _required_ to read all of the
logfile at once, and you are not _unable_ to control
how many data chunks are resident at any time. All the
building blocks for a more robust file reader are
available if you need them. 

So I would say that while this code might not be
something you deploy in production, it still
definitely works as a proof of concept on how to write
efficient code that exploits multicore, etc.

Best,
Thomas



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From bjarne@REDACTED  Sun Oct 28 18:14:19 2007
From: bjarne@REDACTED (=?iso-8859-1?Q?Bjarne_D=E4cker?=)
Date: Sun, 28 Oct 2007 18:14:19 +0100
Subject: [erlang-questions] EUC 2007 oversubscribed
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com><992686.15039.qm@web38809.mail.mud.yahoo.com><d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
Message-ID: <006a01c81986$1a359390$e751d854@Dell>

Registrations to the EUC is overflowing the capacity of our lecture hall.

I would be most grateful if people who have registered but now know that 
they will not be able to come would send me a mail som that I can take them 
off the list and let some late registrants participate instead.

Yours

Bjarne




From mogorman@REDACTED  Sun Oct 28 18:24:14 2007
From: mogorman@REDACTED (Matthew O'Gorman)
Date: Sun, 28 Oct 2007 12:24:14 -0500
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <006a01c81986$1a359390$e751d854@Dell>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
Message-ID: <f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>

all the more reason you should have one stateside preferably in the
south where i live ^_^

mog

On 10/28/07, Bjarne D?cker <bjarne@REDACTED> wrote:
> Registrations to the EUC is overflowing the capacity of our lecture hall.
>
> I would be most grateful if people who have registered but now know that
> they will not be able to come would send me a mail som that I can take them
> off the list and let some late registrants participate instead.
>
> Yours
>
> Bjarne
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From vychodil.hynek@REDACTED  Sun Oct 28 18:32:52 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Sun, 28 Oct 2007 18:32:52 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
Message-ID: <4d08db370710281032j1193cd65i4416d86cc1c0b10a@mail.gmail.com>

Hi Anders,
I rewrote your code a little. I removed all remaining binary bindings
and it is noticeable faster again. Try wf_pichi3.erl.

It requires:
chunk_reader - http://www.erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment-0002.obj
nlt_reader - http://www.erlang.org/pipermail/erlang-questions/attachments/20071028/16fc8af3/attachment-0003.obj
file_map_reduce -
http://www.erlang.org/pipermail/erlang-questions/attachments/20071028/207cb882/attachment.obj

Have a fun
--Hynek (Pichi) Vychodil

On 10/26/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> On 10/23/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > To summarize my progress on the widefinder problem
> > A few days ago I started with Steve Vinoski's tbray16.erl
> > As a baseline on my 1.66 GHz dual core Centrino
> > laptop, Linux,
> > tbray16
> > real    0m7.067s
> > user    0m12.377s
> > sys     0m0.584s
> >
> > I removed the dict used for the shift table,
> > and changed the min_heap_size.
> > That gave
> > real    0m2.713s
> > user    0m4.168s
> > sys     0m0.412s
> >
> > (see tbray_tuple.erl and wfbm4_tuple.erl)
> > Steve reported that it ran in ~1.9 s on his 8 core server.
> >
> > Then I removed the dicts that were used for collecting the
> > matches and used ets instead, and got some improvement
> > on my dual core laptop.
> > real    0m2.220s
> > user    0m3.252s
> > sys     0m0.344s
> >
> > (see tbray_ets.erl and wfbm4_ets.erl)
> >
> > Interestingly Steve reported that it actually performed
> > worse on his 8 core server.
> >
> > These versions all read the whole file into memory at the start.
> > On my laptop that takes ~400ms (when the file is already cached
> > in the OS).
> >
> > So I changed it to read the file in chucks and spawn the worker
> > after each chunk is read.
> >
> > tbray_blockread with 4 processes
> > real    0m1.992s
> > user    0m3.176s
> > sys     0m0.420s
> >
> > (see tbray_blockread.erl and wfbm4_ets.erl)
> >
> > Running it in the erlang shell it takes ~1.8s.
> >
> > Just starting and stopping the VM takes
> > time erl -pa ../../bfile/ebin/ -smp -noshell -run init stop
> >
> > real    0m1.229s
> > user    0m0.208s
> > sys     0m0.020s
> >
> > It would be interesting to see how it runs on other machines,
> > with more cores.
> >
> > /Anders
> >
> >
>
> So I have a new version that I think will break the 1 second barrier
> on Steve's 8-core
> box.
> The best I have seen on my dual core laptop is
> real:  0m1.689s
> user: 0m2.2756s
> sys:  0m0.396s
>
> The changes relative my latest posted tbray_blockread.erl are
> - reading the file is in a separate process
> - never bind variables to sub binaries unless absolutely necessary
> - only have a limited number of worker processes at any time
>
> One lesson from this exercise is that it can be bad for performance,
> the result of changing the code to not bind variables to sub binaries
> can be seen in the garbage collection statistics.
>
> wfinder, (an unreleased version that ran in 1.050s on Steve's 8-core)
> garbage collections: 46302
> words reclaimed: 501768347
>
> wfinder1
> garbage collections: 13917
> words reclaimed: 384561741
>
> /Anders
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wf_pichi3.erl
Type: application/octet-stream
Size: 3858 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/814bacad/attachment.obj>

From michael.campbell@REDACTED  Sun Oct 28 19:41:26 2007
From: michael.campbell@REDACTED (Michael Campbell)
Date: Sun, 28 Oct 2007 14:41:26 -0400
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
Message-ID: <811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>

On 10/28/07, Matthew O'Gorman <mogorman@REDACTED> wrote:
> all the more reason you should have one stateside preferably in the
> south where i live ^_^


There are some really nice convention halls in Atlanta GA and Orlando FL...  =)


irc:MoeD

-- 
Zawinski's Law: Every program attempts to expand until it can read
mail. Those programs which cannot so expand are replaced by ones which
can. -- Jamie Zawinski


From csanto@REDACTED  Sun Oct 28 19:55:06 2007
From: csanto@REDACTED (Corrado Santoro)
Date: Sun, 28 Oct 2007 19:55:06 +0100
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>	<992686.15039.qm@web38809.mail.mud.yahoo.com>	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>	<006a01c81986$1a359390$e751d854@Dell>	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
Message-ID: <4724DB0A.9000903@diit.unict.it>

Michael Campbell wrote:
> There are some really nice convention halls in Atlanta GA and Orlando FL...  =)

... but the weather in Sicily, in November, is still very nice...
(Francesco knows)... and we have very huge halls too ;-)

--Corrado

-- 
==================================================================
Eng. Corrado Santoro, Ph.D.
University of Catania - ITALY - Engineering Faculty

Tel: +39 095 7382380        VoIP: sip:7035@REDACTED

Personal Home Page: http://www.diit.unict.it/users/csanto
     NUXI Home Page: http://nuxi.diit.unict.it
==================================================================




From dking@REDACTED  Sun Oct 28 20:18:04 2007
From: dking@REDACTED (David King)
Date: Sun, 28 Oct 2007 12:18:04 -0700
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
Message-ID: <CB5291EC-C9FC-4669-81F4-4E9304FB0E49@ketralnis.com>

> On 10/28/07, Matthew O'Gorman <mogorman@REDACTED> wrote:
>> all the more reason you should have one stateside preferably in the
>> south where i live ^_^
> There are some really nice convention halls in Atlanta GA and  
> Orlando FL...  =)

And the Bay Area, CA


From erlangx@REDACTED  Sun Oct 28 20:52:17 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Sun, 28 Oct 2007 12:52:17 -0700
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
In-Reply-To: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>
Message-ID: <20071028195217.GT6096@delora.autosys.us>

  see http://autosys.us/misc/vipin.html function foo/2
  also http://autosys.us/misc/vipin.erl

  if I understand correctly your question...

~Michael


On Sun, Oct 28, 2007 at 03:18:28PM +0100, Hynek Vychodil wrote:
> Hello,
> I don't know how to write specification of function with many
> patterns. For example how to specify this simple function
> 
> foo(X, Y) when is_integer(X), is_integer(Y) -> true;
> foo(X, Y) when is_atom(X), is_atom(Y) -> false.
> 
> What should I write when I want to see in documentation that there
> must be both parameters same type. I would like see in documentation
> some like
> 
> foo(X::integer(), Y::integer()) -> true;
> foo(X::atom(), Y::atom()) -> false.
> 
> Best
> -- Hynek (Pichi) Vychodil
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,47249a5450981775114153!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From anders.nygren@REDACTED  Sun Oct 28 23:27:33 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Sun, 28 Oct 2007 17:27:33 -0500
Subject: [erlang-questions] widefinder update
In-Reply-To: <4d08db370710281032j1193cd65i4416d86cc1c0b10a@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
	<4d08db370710281032j1193cd65i4416d86cc1c0b10a@mail.gmail.com>
Message-ID: <d3520a2f0710281527x5eb73ac8va41be0bfc6b4648@mail.gmail.com>

On 10/28/07, Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
> Hi Anders,
> I rewrote your code a little. I removed all remaining binary bindings
> and it is noticeable faster again. Try wf_pichi3.erl.
>

Hynek
that was great, Your change brings my wfinder1_1 + wfbm4_ets1_1
down to
real    0m1.118s
user    0m1.640s
sys     0m0.368s
on my 1.66 GHz dual core laptop.

As a comparison Your wf_pichi3
real    0m1.854s
user    0m2.928s
sys     0m0.336s
on my laptop.

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_ets1_1.erl
Type: text/x-erlang
Size: 2875 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/579863ee/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfinder1_1.erl
Type: text/x-erlang
Size: 5542 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071028/579863ee/attachment-0001.bin>

From vinoski@REDACTED  Mon Oct 29 05:56:05 2007
From: vinoski@REDACTED (Steve Vinoski)
Date: Mon, 29 Oct 2007 00:56:05 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710281527x5eb73ac8va41be0bfc6b4648@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
	<4d08db370710281032j1193cd65i4416d86cc1c0b10a@mail.gmail.com>
	<d3520a2f0710281527x5eb73ac8va41be0bfc6b4648@mail.gmail.com>
Message-ID: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>

On 10/28/07, Anders Nygren <anders.nygren@REDACTED> wrote:
>
> On 10/28/07, Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
> > Hi Anders,
> > I rewrote your code a little. I removed all remaining binary bindings
> > and it is noticeable faster again. Try wf_pichi3.erl.
> >
>
> Hynek
> that was great, Your change brings my wfinder1_1 + wfbm4_ets1_1
> down to
> real    0m1.118s
> user    0m1.640s
> sys     0m0.368s
> on my 1.66 GHz dual core laptop.


And on the 8-core 2.33GHz Intel Xeon Linux box with 2 GB RAM, this version
is extremely fast:

real 0m0.567s
user 0m2.249s
sys 0m0.956s

--steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/f1de3082/attachment.htm>

From vychodil.hynek@REDACTED  Mon Oct 29 08:19:14 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 29 Oct 2007 08:19:14 +0100
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
In-Reply-To: <20071028195217.GT6096@delora.autosys.us>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>
	<20071028195217.GT6096@delora.autosys.us>
Message-ID: <4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>

No, it isn't what I really want. I want something like this:

http://erlang.org/doc/man/ets.html#update_counter/6

But where all parameters changes and only some parameters types works together.

open(File::file_handler(), Procs::integer()) -> {ok, handler()}
open(FileName::list(), BlkSize::integer()) -> {ok, handler()}

But open(File::file_handler(), BlkSize::integer()) and
open(FileName::list(), Procs::integer()) are nonsense. I want write it
in @spec tag and I wouldn't like use open(FilenameOrHandler,
ProcsOrBlkSize) because only two from four combinations have mater.

--Hynek (Pichi) Vychodil

On 10/28/07, Michael McDaniel <erlangx@REDACTED> wrote:
>   see http://autosys.us/misc/vipin.html function foo/2
>   also http://autosys.us/misc/vipin.erl
>
>   if I understand correctly your question...
>
> ~Michael
>
>
> On Sun, Oct 28, 2007 at 03:18:28PM +0100, Hynek Vychodil wrote:
> > Hello,
> > I don't know how to write specification of function with many
> > patterns. For example how to specify this simple function
> >
> > foo(X, Y) when is_integer(X), is_integer(Y) -> true;
> > foo(X, Y) when is_atom(X), is_atom(Y) -> false.
> >
> > What should I write when I want to see in documentation that there
> > must be both parameters same type. I would like see in documentation
> > some like
> >
> > foo(X::integer(), Y::integer()) -> true;
> > foo(X::atom(), Y::atom()) -> false.
> >
> > Best
> > -- Hynek (Pichi) Vychodil
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> > !DSPAM:52,47249a5450981775114153!
> >
> >
>
> --
> Michael McDaniel
> Portland, Oregon, USA
> http://autosys.us
> +1 503 283 5284
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From casper2000a@REDACTED  Mon Oct 29 08:46:29 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Mon, 29 Oct 2007 13:16:29 +0530
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>	<20071028195217.GT6096@delora.autosys.us>
	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
Message-ID: <003601c819ff$d1a1b5d0$74e52170$@com>

Hi,

Is there a Open Source Sigtran stack developed preferably in Erlang? I mean
actively developing and not a stale project. Anybody planning to do such
thing?

If there's none, I am interested to know how many of you would want somebody
to make this initiative? If the interest level is high, I might be able to
convince the relevant people in my office to Open Source the Sigtran stack
we're developing. Let me know your interest.

BRgds,
- Eranga




From thomasl_erlang@REDACTED  Mon Oct 29 09:35:09 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 29 Oct 2007 01:35:09 -0700 (PDT)
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
Message-ID: <639923.2462.qm@web38805.mail.mud.yahoo.com>


--- Steve Vinoski <vinoski@REDACTED> wrote:

> On 10/28/07, Anders Nygren <anders.nygren@REDACTED>
> wrote:
> >
> > On 10/28/07, Hynek Vychodil
> <vychodil.hynek@REDACTED> wrote:
> > > Hi Anders,
> > > I rewrote your code a little. I removed all
> remaining binary bindings
> > > and it is noticeable faster again. Try
> wf_pichi3.erl.
> > >
> >
> > Hynek
> > that was great, Your change brings my wfinder1_1 +
> wfbm4_ets1_1
> > down to
> > real    0m1.118s
> > user    0m1.640s
> > sys     0m0.368s
> > on my 1.66 GHz dual core laptop.
> 
> 
> And on the 8-core 2.33GHz Intel Xeon Linux box with
> 2 GB RAM, this version
> is extremely fast:
> 
> real 0m0.567s
> user 0m2.249s
> sys 0m0.956s

Impressive. However, can someone explain why 
   sys > real? 
Is the kernel too running in parallel?

Best,
Thomas


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From ft@REDACTED  Mon Oct 29 09:45:24 2007
From: ft@REDACTED (Fredrik Thulin)
Date: Mon, 29 Oct 2007 09:45:24 +0100
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
In-Reply-To: <4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>	<20071028195217.GT6096@delora.autosys.us>
	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
Message-ID: <47259DA4.90000@it.su.se>

Hynek Vychodil wrote:
> No, it isn't what I really want. I want something like this:
> 
> http://erlang.org/doc/man/ets.html#update_counter/6
> 
> But where all parameters changes and only some parameters types works together.
> 
> open(File::file_handler(), Procs::integer()) -> {ok, handler()}
> open(FileName::list(), BlkSize::integer()) -> {ok, handler()}
> 
> But open(File::file_handler(), BlkSize::integer()) and
> open(FileName::list(), Procs::integer()) are nonsense. I want write it
> in @spec tag and I wouldn't like use open(FilenameOrHandler,
> ProcsOrBlkSize) because only two from four combinations have mater.

I'd like to suggest that you instead make two different functions,

   open/2
   open_fh/2

or similar... that would both make your interface cleaner (IMO), and 
solve the documentation issue =).

/Fredrik


From Bruce@REDACTED  Mon Oct 29 09:50:02 2007
From: Bruce@REDACTED (Bruce Fitzsimons)
Date: Mon, 29 Oct 2007 21:50:02 +1300
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <003601c819ff$d1a1b5d0$74e52170$@com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>	<20071028195217.GT6096@delora.autosys.us>	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
	<003601c819ff$d1a1b5d0$74e52170$@com>
Message-ID: <47259EBA.5010709@Fitzsimons.org>

Eranga Udesh wrote:
> If there's none, I am interested to know how many of you would want somebody
> to make this initiative? If the interest level is high, I might be able to
> convince the relevant people in my office to Open Source the Sigtran stack
> we're developing. Let me know your interest.
>   
An opensource Erlang SIGTRAN stack would be excellent. Are you thinking 
the full set (M2UA/M2PA/M3UA/SUA/IUA) or a subset? M3UA and SUA would be 
most useful for the applications I see on a daily basis.

Regards,
Bruce


From raimo+erlang-questions@REDACTED  Mon Oct 29 09:52:09 2007
From: raimo+erlang-questions@REDACTED (Raimo Niskanen)
Date: Mon, 29 Oct 2007 09:52:09 +0100
Subject: [erlang-questions] :  idea: tagging subject line in this group
In-Reply-To: <4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>
References: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>
	<4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>
Message-ID: <20071029085209.GB521@erix.ericsson.se>

I am for it.

I can also make it possible to customize your Mailman
subscription in to filter on such topics (it is called
`topic' in Mailman lingo).

The Mailman topic filter will filter the Subject: header
for (topic specifig) regular expressions of my
(listadmin) choice and each user can then choose
to receive messages on the configured topics or not.

The only problem has been to define the topics.
Can we take Joe's suggestion as it is? Topics can
of course be added later, but every time we change
topic definitions it will affect all topic users.

We will also have to be prepared for the noobs that
has not read the welcome message (where I will have
to explain about topics) or the list presentation
(where I also will have to explain it).



On Fri, Oct 26, 2007 at 02:10:07PM -0400, Aaron Feng wrote:
> I think this would be a great idea.  This will also make filtering out
> messages much easier.
> 
> Aaron
> 
> On 10/26/07, Joe Armstrong <erlang@REDACTED> wrote:
> >
> > This group is getting pretty busy. It's also a useful resource for
> > finding out how to get this done.
> >
> > Can I suggest a simple tagging of the subject line. As follows:
> >
> >      idea:    An idea you want to share and get feedback on
> >      how:     A request for technical info
> >
> > Replies are just follow ups to the original posting.
> >
> >     erlounge: announce an erlounge
> >     beginner: a beginners question
> >     clarify:     please clarify ...
> >     program: I've written a fun program ...
> >     ad: an advertisement for something
> >     jobad: a job ad
> >     link: read this article it's interesting
> >
> > /Joe
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >

> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions

-- 

/ Raimo Niskanen, Erlang/OTP, Ericsson AB


From thomasl_erlang@REDACTED  Mon Oct 29 10:11:19 2007
From: thomasl_erlang@REDACTED (Thomas Lindgren)
Date: Mon, 29 Oct 2007 02:11:19 -0700 (PDT)
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
Message-ID: <233934.61038.qm@web38810.mail.mud.yahoo.com>


--- Steve Vinoski <vinoski@REDACTED> wrote:

> On 10/28/07, Anders Nygren <anders.nygren@REDACTED>
> wrote:
> >
> > On 10/28/07, Hynek Vychodil
> <vychodil.hynek@REDACTED> wrote:
> > > Hi Anders,
> > > I rewrote your code a little. I removed all
> remaining binary bindings
> > > and it is noticeable faster again. Try
> wf_pichi3.erl.
> > >
> >
> > Hynek
> > that was great, Your change brings my wfinder1_1 +
> wfbm4_ets1_1
> > down to
> > real    0m1.118s
> > user    0m1.640s
> > sys     0m0.368s
> > on my 1.66 GHz dual core laptop.
> 
> 
> And on the 8-core 2.33GHz Intel Xeon Linux box with
> 2 GB RAM, this version
> is extremely fast:
> 
> real 0m0.567s
> user 0m2.249s
> sys 0m0.956s

(I'll ignore the unexplained sys time below. That
makes the discussion a bit preliminary; perhaps the
derived results should be computed some other way.
Apply grain of salt appropriately.)

For those keeping track, the latest result is fully
2.7 times faster than the best previous version (which
was block read), and 17.3 times faster than the
initial version. The latest speedup is basically due
to doing less work. However, note that user time fell
by a somewhat greater ratio than real time, which
might mean parallelization overheads are becoming
visible.

Also, the user time of 2.249 seconds is now close to
the Ruby user time, which were 2.095s on the same
hardware, while the Erlang parallelization speedup
(user/real) on top of this is 3.95 out of 8. Comparing
the real times of Erlang (0.567s) and Ruby (2.21s), we
get about the same execution time speedup, 3.9. Not
too shabby, huh?

Is there anything more to be wrung out of this
program? Well, apart from further tuning, one can note
that Ruby had 0.1s sys time, while Erlang apparently
needs a bit more, 0.5-1.0s. Why?

Taking a wider view, it would also be very interesting
to see how to apply these lessons to more general
problems and/or libraries.

Best,
Thomas



__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From casper2000a@REDACTED  Mon Oct 29 10:26:31 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Mon, 29 Oct 2007 14:56:31 +0530
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <47259EBA.5010709@Fitzsimons.org>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>	<20071028195217.GT6096@delora.autosys.us>	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
	<003601c819ff$d1a1b5d0$74e52170$@com>
	<47259EBA.5010709@Fitzsimons.org>
Message-ID: <005d01c81a0d$ca9a0e50$5fce2af0$@com>

Thanks for your response.

Initially it'll be a subset of the full Sigtran stack (M2PA & M3UA). However
the development plan is for the full stack. Are you already using M3UA and
SUA stacks? Are they in Erlang or licensed from a 3rd party?

BRgds,
- Eranga



-----Original Message-----
From: Bruce Fitzsimons [mailto:Bruce@REDACTED] 
Sent: Monday, October 29, 2007 2:20 PM
To: Eranga Udesh
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Open Source ErlSigtran

Eranga Udesh wrote:
> If there's none, I am interested to know how many of you would want
somebody
> to make this initiative? If the interest level is high, I might be able to
> convince the relevant people in my office to Open Source the Sigtran stack
> we're developing. Let me know your interest.
>   
An opensource Erlang SIGTRAN stack would be excellent. Are you thinking 
the full set (M2UA/M2PA/M3UA/SUA/IUA) or a subset? M3UA and SUA would be 
most useful for the applications I see on a daily basis.

Regards,
Bruce




From richardc@REDACTED  Mon Oct 29 11:31:03 2007
From: richardc@REDACTED (Richard Carlsson)
Date: Mon, 29 Oct 2007 11:31:03 +0100
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
In-Reply-To: <4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>	<20071028195217.GT6096@delora.autosys.us>
	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
Message-ID: <4725B667.5070303@it.uu.se>

Hynek Vychodil wrote:
> No, it isn't what I really want. I want something like this:
> 
> http://erlang.org/doc/man/ets.html#update_counter/6
> 
> But where all parameters changes and only some parameters types works together.
> 
> open(File::file_handler(), Procs::integer()) -> {ok, handler()}
> open(FileName::list(), BlkSize::integer()) -> {ok, handler()}
> 
> But open(File::file_handler(), BlkSize::integer()) and
> open(FileName::list(), Procs::integer()) are nonsense. I want write it
> in @spec tag and I wouldn't like use open(FilenameOrHandler,
> ProcsOrBlkSize) because only two from four combinations have mater.

EDoc doesn't support this style of documentation, and that is a design
decision. The OTP docs are (at least traditionally) written in a more
or less free-form style, and nobody really expects to be able to read
them automatically and use those specifications for things like type
analysis - they are often a bit too "loose" and require a human to
interpret their meaning.

The example you point to, update_counter/6, could easily be described
using normal edoc syntax, since it actually returns the same type
regardless of the exact type of its parameters:

  @spec update_counter(Tab, Key, How) -> Result
    where
         How = {Pos,Incr,Threshold,SetValue} | {Pos,Incr} | Incr,
         ...
         Result = integer()

In almost all cases when you feel the need to use this kind of
documentation, where the output type depends on the input type:
   foo(integer()) -> integer();
   foo(atom()) -> atom().

it means that you are doing the wrong thing, and that you should
probably redesign your interface. (Trust me on this one.)

     /Richard



From vychodil.hynek@REDACTED  Mon Oct 29 11:35:05 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 29 Oct 2007 11:35:05 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <233934.61038.qm@web38810.mail.mud.yahoo.com>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
Message-ID: <4d08db370710290335s2afa0f53sc4cacc0f4a508b6d@mail.gmail.com>

Ruby code is slower on my old single core home desktop and sys time
are almost same.

http://pichis-blog.blogspot.com/2007/10/faster-than-ruby-but-scalable.html

--Hynek (Pichi) Vychodil

On 10/29/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> > On 10/28/07, Anders Nygren <anders.nygren@REDACTED>
> > wrote:
> > >
> > > On 10/28/07, Hynek Vychodil
> > <vychodil.hynek@REDACTED> wrote:
> > > > Hi Anders,
> > > > I rewrote your code a little. I removed all
> > remaining binary bindings
> > > > and it is noticeable faster again. Try
> > wf_pichi3.erl.
> > > >
> > >
> > > Hynek
> > > that was great, Your change brings my wfinder1_1 +
> > wfbm4_ets1_1
> > > down to
> > > real    0m1.118s
> > > user    0m1.640s
> > > sys     0m0.368s
> > > on my 1.66 GHz dual core laptop.
> >
> >
> > And on the 8-core 2.33GHz Intel Xeon Linux box with
> > 2 GB RAM, this version
> > is extremely fast:
> >
> > real 0m0.567s
> > user 0m2.249s
> > sys 0m0.956s
>
> (I'll ignore the unexplained sys time below. That
> makes the discussion a bit preliminary; perhaps the
> derived results should be computed some other way.
> Apply grain of salt appropriately.)
>
> For those keeping track, the latest result is fully
> 2.7 times faster than the best previous version (which
> was block read), and 17.3 times faster than the
> initial version. The latest speedup is basically due
> to doing less work. However, note that user time fell
> by a somewhat greater ratio than real time, which
> might mean parallelization overheads are becoming
> visible.
>
> Also, the user time of 2.249 seconds is now close to
> the Ruby user time, which were 2.095s on the same
> hardware, while the Erlang parallelization speedup
> (user/real) on top of this is 3.95 out of 8. Comparing
> the real times of Erlang (0.567s) and Ruby (2.21s), we
> get about the same execution time speedup, 3.9. Not
> too shabby, huh?
>
> Is there anything more to be wrung out of this
> program? Well, apart from further tuning, one can note
> that Ruby had 0.1s sys time, while Erlang apparently
> needs a bit more, 0.5-1.0s. Why?
>
> Taking a wider view, it would also be very interesting
> to see how to apply these lessons to more general
> problems and/or libraries.
>
> Best,
> Thomas
>
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From vychodil.hynek@REDACTED  Mon Oct 29 11:52:41 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 29 Oct 2007 11:52:41 +0100
Subject: [erlang-questions] beginer: How to specify multifunction in edoc
In-Reply-To: <4725B667.5070303@it.uu.se>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>
	<20071028195217.GT6096@delora.autosys.us>
	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
	<4725B667.5070303@it.uu.se>
Message-ID: <4d08db370710290352oc04ef0q6c67123366e62275@mail.gmail.com>

On 10/29/07, Richard Carlsson <richardc@REDACTED> wrote:
> Hynek Vychodil wrote:
> > No, it isn't what I really want. I want something like this:
> >
> > http://erlang.org/doc/man/ets.html#update_counter/6
> >
> > But where all parameters changes and only some parameters types works together.
> >
> > open(File::file_handler(), Procs::integer()) -> {ok, handler()}
> > open(FileName::list(), BlkSize::integer()) -> {ok, handler()}
> >
> > But open(File::file_handler(), BlkSize::integer()) and
> > open(FileName::list(), Procs::integer()) are nonsense. I want write it
> > in @spec tag and I wouldn't like use open(FilenameOrHandler,
> > ProcsOrBlkSize) because only two from four combinations have mater.
>
> EDoc doesn't support this style of documentation, and that is a design
> decision. The OTP docs are (at least traditionally) written in a more
> or less free-form style, and nobody really expects to be able to read
> them automatically and use those specifications for things like type
> analysis - they are often a bit too "loose" and require a human to
> interpret their meaning.
>
> The example you point to, update_counter/6, could easily be described
> using normal edoc syntax, since it actually returns the same type
> regardless of the exact type of its parameters:
>
>   @spec update_counter(Tab, Key, How) -> Result
>     where
>          How = {Pos,Incr,Threshold,SetValue} | {Pos,Incr} | Incr,
>          ...
>          Result = integer()
>
> In almost all cases when you feel the need to use this kind of
> documentation, where the output type depends on the input type:
>    foo(integer()) -> integer();
>    foo(atom()) -> atom().
>
> it means that you are doing the wrong thing, and that you should
> probably redesign your interface. (Trust me on this one.)
>
>      /Richard
>
>

I have varying input parameters but coupling types between positions
and same result type. It's a little different, but you are right, it
can be good argument to rework interface too. It isn't so important
that if result type depend of input type. It is case when second
parameter meaning (not type) depends on first parameter type (and
meaning also). You right that redesign should make more clean
interface.

--Hynek (Pichi) Vychodil


From vychodil.hynek@REDACTED  Mon Oct 29 11:58:37 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Mon, 29 Oct 2007 11:58:37 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
References: <d3520a2f0710231203i3dcb3f28xd784f3ce6ddef2d4@mail.gmail.com>
	<d3520a2f0710261504v5468071ag65f58b2c163170be@mail.gmail.com>
	<4d08db370710281032j1193cd65i4416d86cc1c0b10a@mail.gmail.com>
	<d3520a2f0710281527x5eb73ac8va41be0bfc6b4648@mail.gmail.com>
	<65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
Message-ID: <4d08db370710290358m48a5cccs17c16273cab90f25@mail.gmail.com>

On 10/29/07, Steve Vinoski <vinoski@REDACTED> wrote:
> On 10/28/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> > On 10/28/07, Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
> > > Hi Anders,
> > > I rewrote your code a little. I removed all remaining binary bindings
> > > and it is noticeable faster again. Try wf_pichi3.erl.
> > >
> >
> > Hynek
> > that was great, Your change brings my wfinder1_1 + wfbm4_ets1_1
> > down to
> > real    0m1.118s
> > user    0m1.640s
> > sys     0m0.368s
> > on my 1.66 GHz dual core laptop.
>
>
> And on the 8-core 2.33GHz Intel Xeon Linux box with 2 GB RAM, this version
> is extremely fast:
>
> real 0m0.567s
> user 0m2.249s
> sys 0m0.956s
>  --steve

Did you test wfinder1_1 + wfbm4_ets1_1 or wf_pichi3 + file_map_reduce
+nlt_reader +chunk_reader too? I think nlt_reader should be less
memory consuming than reader from wfinder1_1 and it can cause less sys
time and paralel block splitting can cause some speed up on 8 core,
but I can't test it.

--Hynek (Pichi) Vychodil


From erlang@REDACTED  Mon Oct 29 12:07:08 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Mon, 29 Oct 2007 12:07:08 +0100
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <4724DB0A.9000903@diit.unict.it>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<4724DB0A.9000903@diit.unict.it>
Message-ID: <9b08084c0710290407g57010d9ft770511831d370e40@mail.gmail.com>

But the cold rain in Stockholm in November is very good - It makes you want to
stay indoors in the nice cosy conference room.

If you're very luck you might get sn?blandat regn (snow mixed with
rain) a special
treat!

/Joe Armstrong


On 10/28/07, Corrado Santoro <csanto@REDACTED> wrote:
> Michael Campbell wrote:
> > There are some really nice convention halls in Atlanta GA and Orlando FL...  =)
>
> ... but the weather in Sicily, in November, is still very nice...
> (Francesco knows)... and we have very huge halls too ;-)
>
> --Corrado
>
> --
> ==================================================================
> Eng. Corrado Santoro, Ph.D.
> University of Catania - ITALY - Engineering Faculty
>
> Tel: +39 095 7382380        VoIP: sip:7035@REDACTED
>
> Personal Home Page: http://www.diit.unict.it/users/csanto
>      NUXI Home Page: http://nuxi.diit.unict.it
> ==================================================================
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From dcaoyuan@REDACTED  Mon Oct 29 12:21:12 2007
From: dcaoyuan@REDACTED (Caoyuan)
Date: Mon, 29 Oct 2007 19:21:12 +0800
Subject: [erlang-questions] widefinder update
In-Reply-To: <639923.2462.qm@web38805.mail.mud.yahoo.com>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<639923.2462.qm@web38805.mail.mud.yahoo.com>
Message-ID: <ca96bd270710290421w42b0dc7bs741cdeb15c330bf1@mail.gmail.com>

I think the system time is mostly the disk/io time.

On 10/29/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> > On 10/28/07, Anders Nygren <anders.nygren@REDACTED>
> > wrote:
> > >
> > > On 10/28/07, Hynek Vychodil
> > <vychodil.hynek@REDACTED> wrote:
> > > > Hi Anders,
> > > > I rewrote your code a little. I removed all
> > remaining binary bindings
> > > > and it is noticeable faster again. Try
> > wf_pichi3.erl.
> > > >
> > >
> > > Hynek
> > > that was great, Your change brings my wfinder1_1 +
> > wfbm4_ets1_1
> > > down to
> > > real    0m1.118s
> > > user    0m1.640s
> > > sys     0m0.368s
> > > on my 1.66 GHz dual core laptop.
> >
> >
> > And on the 8-core 2.33GHz Intel Xeon Linux box with
> > 2 GB RAM, this version
> > is extremely fast:
> >
> > real 0m0.567s
> > user 0m2.249s
> > sys 0m0.956s
>
> Impressive. However, can someone explain why
>    sys > real?
> Is the kernel too running in parallel?
>
> Best,
> Thomas
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


-- 
- Caoyuan


From anders.nygren@REDACTED  Mon Oct 29 12:59:41 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 05:59:41 -0600
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <9b08084c0710290407g57010d9ft770511831d370e40@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<4724DB0A.9000903@diit.unict.it>
	<9b08084c0710290407g57010d9ft770511831d370e40@mail.gmail.com>
Message-ID: <d3520a2f0710290459l35745a6aue8fe303e56cc0b84@mail.gmail.com>

On 10/29/07, Joe Armstrong <erlang@REDACTED> wrote:
> But the cold rain in Stockholm in November is very good - It makes you want to
> stay indoors in the nice cosy conference room.
>
> If you're very luck you might get sn?blandat regn (snow mixed with
> rain) a special
> treat!
>

And if You are really lucky it will fall horizontally.

-Anders


> /Joe Armstrong
>
>
> On 10/28/07, Corrado Santoro <csanto@REDACTED> wrote:
> > Michael Campbell wrote:
> > > There are some really nice convention halls in Atlanta GA and Orlando FL...  =)
> >
> > ... but the weather in Sicily, in November, is still very nice...
> > (Francesco knows)... and we have very huge halls too ;-)
> >
> > --Corrado
> >
> > --
> > ==================================================================
> > Eng. Corrado Santoro, Ph.D.
> > University of Catania - ITALY - Engineering Faculty
> >
> > Tel: +39 095 7382380        VoIP: sip:7035@REDACTED
> >
> > Personal Home Page: http://www.diit.unict.it/users/csanto
> >      NUXI Home Page: http://nuxi.diit.unict.it
> > ==================================================================
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
> >
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From me@REDACTED  Mon Oct 29 13:16:03 2007
From: me@REDACTED (KatolaZ)
Date: Mon, 29 Oct 2007 13:16:03 +0100
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <4724DB0A.9000903@diit.unict.it>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<4724DB0A.9000903@diit.unict.it>
Message-ID: <20071029121603.GD13782@katolaz.homeunix.net>

On Sun, Oct 28, 2007 at 07:55:06PM +0100, Corrado Santoro wrote:
> Michael Campbell wrote:
> > There are some really nice convention halls in Atlanta GA and Orlando FL...  =)
> 
> ... but the weather in Sicily, in November, is still very nice...
> (Francesco knows)... and we have very huge halls too ;-)
> 

mmhhhh.... sounds like a proposal for 2008 ? :-) Anyway, would be nice
to host EUC 2008 here in Catania: 

- sea and sun instead of cold and snow ;-)
- Erlounge in a typical sicilian restaunant, with typical sicilian food
- A 300-seats Aula Magna 
- On top of an active volcano in less then 1 hour and half

Sounds good, doesn't it ?

HND

Enzo

-- 
[ Enzo Nicosia aka KatolaZ --- GLUG Catania -- Freaknet Medialab ]
[ me [at] katolaz.homeunix.net -- http://katolaz.homeunix.net -- ]
[ GNU/Linux User:#325780/ICQ UIN: #258332181/GPG key ID 0B5F062F ]
[ Fingerprint: 8E59 D6AA 445E FDB4 A153 3D5A 5F20 B3AE 0B5F 062F ]


From davidnwelton@REDACTED  Mon Oct 29 13:27:59 2007
From: davidnwelton@REDACTED (David Welton)
Date: Mon, 29 Oct 2007 13:27:59 +0100
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <20071029121603.GD13782@katolaz.homeunix.net>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<4724DB0A.9000903@diit.unict.it>
	<20071029121603.GD13782@katolaz.homeunix.net>
Message-ID: <9877cd600710290527r7daf5afer6dc404d6dc4ff749@mail.gmail.com>

> mmhhhh.... sounds like a proposal for 2008 ? :-) Anyway, would be nice
> to host EUC 2008 here in Catania:
>
> - sea and sun instead of cold and snow ;-)

You can get snow too - it's just located in its proper place - towards
the top of the mountain, not at sea level.

> - Erlounge in a typical sicilian restaunant, with typical sicilian food

And wine!

> - A 300-seats Aula Magna
> - On top of an active volcano in less then 1 hour and half

"As part of our demonstration of fault tolerance ..."

> Sounds good, doesn't it ?

Sicily is also cheaper than Sweden.  I've flown into Catania before,
but don't really remember how well connected it is with the rest of
Europe... ?

In any case, I can heartily recommend Sicily as a beautiful place,
with a lot of things to do.

-- 
David N. Welton
http://www.welton.it/davidw/


From ulf.wiger@REDACTED  Mon Oct 29 13:52:45 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Mon, 29 Oct 2007 13:52:45 +0100
Subject: [erlang-questions] :  idea: tagging subject line in this group
In-Reply-To: <20071029085209.GB521@erix.ericsson.se>
References: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>	<4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>
	<20071029085209.GB521@erix.ericsson.se>
Message-ID: <4725D79D.2070709@ericsson.com>

Raimo Niskanen wrote:
> 
> The only problem has been to define the topics.
> Can we take Joe's suggestion as it is?

Joe's suggestions being:

>>> Can I suggest a simple tagging of the subject line. As follows:
>>>
>>>      idea:    An idea you want to share and get feedback on
>>>      how:     A request for technical info
>>>
>>> Replies are just follow ups to the original posting.
>>>
>>>     erlounge: announce an erlounge
>>>     beginner: a beginners question
>>>     clarify:     please clarify ...
>>>     program: I've written a fun program ...
>>>     ad: an advertisement for something
>>>     jobad: a job ad
>>>     link: read this article it's interesting

I think they're fine. I'd prefer 'job' instead of 'jobad',
but that's just me.

I think it's a very good idea.

BR,
Ulf W


From vladdu55@REDACTED  Mon Oct 29 13:53:44 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Mon, 29 Oct 2007 13:53:44 +0100
Subject: [erlang-questions] idea: using erlc for more file types
Message-ID: <95be1d3b0710290553p34a7d96ct90c22ef6c9fbdf5f@mail.gmail.com>

Hi all,

 ("idea" was the closest matching tag, sometimes it's difficult to
know what's appropriate)

I would need to process files in Erlang that are not in the set that
is handled by erlc, but it would be very handy if they were. I think
then that it's not an earth-shattering suggestion that erlc should be
able to handle other file types.

A way to solve that would be by adding a command-line parameter
  -compiler mymodule:myfunction
and use the specified function as entry point for the compiler proper.

A patch is attached (but of course, better and cleaner solutions may exist).

best regards,
Vlad

----------------------

Index: erts/etc/common/erlc.c
===================================================================
--- erts/etc/common/erlc.c	(revision 18)
+++ erts/etc/common/erlc.c	(working copy)
@@ -187,6 +187,9 @@
 		PUSH2("@output_type", output_type);
 		break;
 	    case 'c':		/* Allowed for compatibility with 'erl'. */
+		if (strcmp(argv[1], "-compiler") == 0) {
+		    PUSH2("@compiler", process_opt(&argc, &argv, 0));
+		} else
 		if (strcmp(argv[1], "-compile") != 0)
 		    goto error;
 		break;
Index: lib/stdlib/include/erl_compile.hrl
===================================================================
--- lib/stdlib/include/erl_compile.hrl	(revision 18)
+++ lib/stdlib/include/erl_compile.hrl	(working copy)
@@ -36,7 +36,8 @@
 	  specific=[],				% Compiler specific options.
 	  outfile="",				% Name of output file (internal
 						% use in erl_compile.erl).
-	  cwd					% Current working directory
+	  cwd,					% Current working directory
 						% for erlc.
+	  compiler
 	 }).

Index: lib/stdlib/src/erl_compile.erl
===================================================================
--- lib/stdlib/src/erl_compile.erl	(revision 18)
+++ lib/stdlib/src/erl_compile.erl	(working copy)
@@ -26,22 +26,24 @@

 %% Mapping from extension to {M,F} to run the correct compiler.

-compiler(".erl") ->    {compile,         compile};
-compiler(".S") ->      {compile,         compile_asm};
-compiler(".beam") ->   {compile,         compile_beam};
-compiler(".core") ->   {compile,         compile_core};
-compiler(".mib") ->    {snmpc,           compile};
-compiler(".bin") ->    {snmpc,           mib_to_hrl};
-compiler(".yrl") ->    {yecc,            compile};
-compiler(".script") -> {systools,        script2boot};
-compiler(".rel") ->    {systools,        compile_rel};
-compiler(".idl") ->    {ic,              compile};
-compiler(".asn1") ->   {asn1ct,          compile_asn1};
-compiler(".asn") ->    {asn1ct,          compile_asn};
-compiler(".py") ->     {asn1ct,          compile_py};
-compiler(".xml") ->    {xmerl_scan,      process};
-compiler(_) ->         no.
+compiler(".erl", _) ->    {compile,         compile};
+compiler(".S", _) ->      {compile,         compile_asm};
+compiler(".beam", _) ->   {compile,         compile_beam};
+compiler(".core", _) ->   {compile,         compile_core};
+compiler(".mib", _) ->    {snmpc,           compile};
+compiler(".bin", _) ->    {snmpc,           mib_to_hrl};
+compiler(".yrl", _) ->    {yecc,            compile};
+compiler(".script", _) -> {systools,        script2boot};
+compiler(".rel", _) ->    {systools,        compile_rel};
+compiler(".idl", _) ->    {ic,              compile};
+compiler(".asn1", _) ->   {asn1ct,          compile_asn1};
+compiler(".asn", _) ->    {asn1ct,          compile_asn};
+compiler(".py", _) ->     {asn1ct,          compile_py};
+compiler(".xml", _) ->    {xmerl_scan,      process};
+compiler(_, #options{compiler=C}) -> C;
+compiler(_, _) ->         no.

+
 %% Entry from command line.

 compile_cmdline(List) ->
@@ -130,8 +132,16 @@
     compile1(Rest, Cwd, Opts#options{output_type=OutputType});
 compile1(['@files'|Rest], Cwd, Opts) ->
     Includes = lists:reverse(Opts#options.includes),
-    compile2(Rest, Cwd, Opts#options{includes=Includes}).
+    compile2(Rest, Cwd, Opts#options{includes=Includes});
+compile1(['@compiler', MF|Rest], Cwd, Opts) ->
+    case string:tokens(MF, ":") of
+	  [M, F] ->
+		compile2(Rest, Cwd, Opts#options{compiler={make_term(M), make_term(F)}});
+	  _ ->
+		compile2(Rest, Cwd, Opts)
+	end.

+
 compile2(Files, Cwd, Opts) ->
     case {Opts#options.outfile, length(Files)} of
 	{"", _} ->


From csanto@REDACTED  Mon Oct 29 14:06:32 2007
From: csanto@REDACTED (Corrado Santoro)
Date: Mon, 29 Oct 2007 14:06:32 +0100
Subject: [erlang-questions] idea: EUC 2007 oversubscribed
In-Reply-To: <9877cd600710290527r7daf5afer6dc404d6dc4ff749@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>	<992686.15039.qm@web38809.mail.mud.yahoo.com>	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>	<006a01c81986$1a359390$e751d854@Dell>	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>	<4724DB0A.9000903@diit.unict.it>	<20071029121603.GD13782@katolaz.homeunix.net>
	<9877cd600710290527r7daf5afer6dc404d6dc4ff749@mail.gmail.com>
Message-ID: <4725DAD8.7030701@diit.unict.it>

David Welton wrote:
> Sicily is also cheaper than Sweden.  I've flown into Catania before,
> but don't really remember how well connected it is with the rest of
> Europe... ?
Catania Fontanarossa is the fourth airport in Italy. It has direct 
connections with Rome and Milano, and also direct connections with many 
European cities (London, Frankfurt, Munchen, Wien, Paris, Barcelona, etc.).

--Corrado

-- 
==================================================================
Eng. Corrado Santoro, Ph.D.
University of Catania - ITALY - Engineering Faculty

Tel: +39 095 7382380        VoIP: sip:7035@REDACTED

Personal Home Page: http://www.diit.unict.it/users/csanto
     NUXI Home Page: http://nuxi.diit.unict.it
==================================================================



From vladdu55@REDACTED  Mon Oct 29 14:04:18 2007
From: vladdu55@REDACTED (Vlad Dumitrescu)
Date: Mon, 29 Oct 2007 14:04:18 +0100
Subject: [erlang-questions] idea: using erlc for more file types
In-Reply-To: <95be1d3b0710290553p34a7d96ct90c22ef6c9fbdf5f@mail.gmail.com>
References: <95be1d3b0710290553p34a7d96ct90c22ef6c9fbdf5f@mail.gmail.com>
Message-ID: <95be1d3b0710290604r6aaa5a83k90228d90d31ccc71@mail.gmail.com>

Hmm, something is missing from the patch:

Index: erl_compile.erl
===================================================================
--- erl_compile.erl	(revision 20)
+++ erl_compile.erl	(working copy)
@@ -180,7 +180,7 @@
     io:format("File has no extension: ~s~n", [Input]),
     error;
 compile_file(Ext, Input, Output, Options) ->
-    case compiler(Ext) of
+    case compiler(Ext, Options) of
 	no ->
 	    io:format("Unknown extension: '~s'\n", [Ext]),
 	    error;


From matthias@REDACTED  Mon Oct 29 13:19:09 2007
From: matthias@REDACTED (Matthias Lang)
Date: Mon, 29 Oct 2007 13:19:09 +0100
Subject: [erlang-questions] is anyone else using erlang docbuilder?
In-Reply-To: <4721DD8A.5020608@lshift.net>
References: <18209.52055.446607.298180@cors.corelatus.se>
	<4721DD8A.5020608@lshift.net>
Message-ID: <18213.53181.697124.903507@cors.corelatus.se>

Tony Garnock-Jones writes:

 > Was it SGML docbook? Is docbuilder XML?

Apparently docbook can be either SGML or XML. My source was
SGML. Docbuilder is XML.

 > Would a CDATA block make things easier?
 > 
 > <code><![CDATA[
 >   <0.23.0>
 > ]]></code>

Yes! Thanks! That solves the problem.

Matthias


From jao@REDACTED  Mon Oct 29 13:50:02 2007
From: jao@REDACTED (Jack Orenstein)
Date: Mon, 29 Oct 2007 08:50:02 -0400
Subject: [erlang-questions] Erlang implementations for deployment
Message-ID: <2D35C26B-76FF-47A7-A871-44B6F83F7D6F@geophile.com>

Do production systems tend to be deployed using the open source  
implementation of Erlang? Or is Ericsson's Commercial Erlang more  
common? I'm particularly interested in running Erlang on Linux.

Jack Orenstein


From headspin@REDACTED  Mon Oct 29 14:24:13 2007
From: headspin@REDACTED (dda)
Date: Mon, 29 Oct 2007 21:24:13 +0800
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <4724DB0A.9000903@diit.unict.it>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<4724DB0A.9000903@diit.unict.it>
Message-ID: <eb8fc1930710290624iae32ab8x156d4fd216466d83@mail.gmail.com>

Well if weather is a concern, you wimps ;-), Hong Kong is an
appropriate venue then! It's 26? C today, at 21:22. There *are*
appropriate conference facilities, good food, and proselytising
outside the Western world would do Erlang some good!

-- 
Didier

On 10/29/07, Corrado Santoro <csanto@REDACTED> wrote:
> Michael Campbell wrote:
> > There are some really nice convention halls in Atlanta GA and Orlando FL...  =)
>
> ... but the weather in Sicily, in November, is still very nice...
> (Francesco knows)... and we have very huge halls too ;-)
>
> --Corrado
>
> --
> ==================================================================
> Eng. Corrado Santoro, Ph.D.
> University of Catania - ITALY - Engineering Faculty
>
> Tel: +39 095 7382380        VoIP: sip:7035@REDACTED
>
> Personal Home Page: http://www.diit.unict.it/users/csanto
>      NUXI Home Page: http://nuxi.diit.unict.it
> ==================================================================
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>

From gordonguthrie@REDACTED  Mon Oct 29 15:07:34 2007
From: gordonguthrie@REDACTED (Gordon Guthrie)
Date: Mon, 29 Oct 2007 14:07:34 -0000 (GMT)
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <eb8fc1930710290624iae32ab8x156d4fd216466d83@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com><992686.15039.qm@web38809.mail.mud.yahoo.com><d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com><4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com><006a01c81986$1a359390$e751d854@Dell><f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com><811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com><4724DB0A.9000903@diit.unict.it>
	<eb8fc1930710290624iae32ab8x156d4fd216466d83@mail.gmail.com>
Message-ID: <64095.80.0.63.43.1193666854.squirrel@backawinner.gg>

I would chip with Bonnie Scotland but the last big techie conference to
come  here - Debian - decided to be very local friendly and commission
their own tartan for the event

But unfortunately it turned out not to be a tartan at all (tartan has both
horizontal and vertical symmetry in the pattern and the  Debian boys hid
Debian in Morse in theirs but not palindromically...

Some people might regard that as a metaphor for open source, but I
couldn't possible comment...

So if you promise to play well, you'd be very welcome...

Gordon

> Well if weather is a concern, you wimps ;-), Hong Kong is an
> appropriate venue then! It's 26?? C today, at 21:22. There *are*
> appropriate conference facilities, good food, and proselytising
> outside the Western world would do Erlang some good!
>
> --
> Didier
>
> On 10/29/07, Corrado Santoro <csanto@REDACTED> wrote:
>> Michael Campbell wrote:
>> > There are some really nice convention halls in Atlanta GA and Orlando
>> FL...  =)
>>
>> ... but the weather in Sicily, in November, is still very nice...
>> (Francesco knows)... and we have very huge halls too ;-)
>>
>> --Corrado
>>
>> --
>> ==================================================================
>> Eng. Corrado Santoro, Ph.D.
>> University of Catania - ITALY - Engineering Faculty
>>
>> Tel: +39 095 7382380        VoIP: sip:7035@REDACTED
>>
>> Personal Home Page: http://www.diit.unict.it/users/csanto
>>      NUXI Home Page: http://nuxi.diit.unict.it
>> ==================================================================
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From erlangx@REDACTED  Mon Oct 29 15:19:54 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Mon, 29 Oct 2007 07:19:54 -0700
Subject: [erlang-questions] :  idea: tagging subject line in this group
In-Reply-To: <4725D79D.2070709@ericsson.com>
References: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>
	<4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>
	<20071029085209.GB521@erix.ericsson.se>
	<4725D79D.2070709@ericsson.com>
Message-ID: <20071029141954.GI21717@delora.autosys.us>

On Mon, Oct 29, 2007 at 01:52:45PM +0100, Ulf Wiger (TN/EAB) wrote:
> Raimo Niskanen wrote:
> > 
> > The only problem has been to define the topics.
> > Can we take Joe's suggestion as it is?
> 
> Joe's suggestions being:
> 
> >>> Can I suggest a simple tagging of the subject line. As follows:
> >>>
> >>>      idea:    An idea you want to share and get feedback on
> >>>      how:     A request for technical info
> >>>
> >>> Replies are just follow ups to the original posting.
> >>>
> >>>     erlounge: announce an erlounge
> >>>     beginner: a beginners question
> >>>     clarify:     please clarify ...
> >>>     program: I've written a fun program ...
> >>>     ad: an advertisement for something
> >>>     jobad: a job ad
> >>>     link: read this article it's interesting
> 
> I think they're fine. I'd prefer 'job' instead of 'jobad',
> but that's just me.
> 
> I think it's a very good idea.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 I agree, and I also prefer 'job'.
~Michael

> 
> BR,
> Ulf W
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,4725d80850983570250697!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From kenneth.lundin@REDACTED  Mon Oct 29 15:37:07 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Mon, 29 Oct 2007 15:37:07 +0100
Subject: [erlang-questions] Erlang implementations for deployment
In-Reply-To: <2D35C26B-76FF-47A7-A871-44B6F83F7D6F@geophile.com>
References: <2D35C26B-76FF-47A7-A871-44B6F83F7D6F@geophile.com>
Message-ID: <de422a40710290737h61436c47s6ba5922ebd479294@mail.gmail.com>

Hi,

The term Commercial Erlang is out dated. There used to be be a
"commercial" version with some extra components compared with the Open
Source version but that was 10 years ago or so.
Today there is only one version of Erlang OTP and that is what we
deliver as Open Source Erlang.

When it comes to support we (Ericsson Erlang/OTP team) can
unfortunately not deliver new support license contracts because of
administrative reasons (has been like that several years now) but we
are working on it and will hopefully in due time come to an
arrangement that works.

Erlang on Linux is a good choice, Linux x86 is probably the most used
platform for deployment of systems written in Erlang.

/Kenneth Erlang/OTP team at Ericsson

On 10/29/07, Jack Orenstein <jao@REDACTED> wrote:
> Do production systems tend to be deployed using the open source
> implementation of Erlang? Or is Ericsson's Commercial Erlang more
> common? I'm particularly interested in running Erlang on Linux.
>
> Jack Orenstein
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From anders.nygren@REDACTED  Mon Oct 29 16:25:02 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 09:25:02 -0600
Subject: [erlang-questions] widefinder update
In-Reply-To: <233934.61038.qm@web38810.mail.mud.yahoo.com>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
Message-ID: <d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>

On 10/29/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
> --- Steve Vinoski <vinoski@REDACTED> wrote:
>
> > On 10/28/07, Anders Nygren <anders.nygren@REDACTED>
> > wrote:
> > >
> > > On 10/28/07, Hynek Vychodil
> > <vychodil.hynek@REDACTED> wrote:
> > > > Hi Anders,
> > > > I rewrote your code a little. I removed all
> > remaining binary bindings
> > > > and it is noticeable faster again. Try
> > wf_pichi3.erl.
> > > >
> > >
> > > Hynek
> > > that was great, Your change brings my wfinder1_1 +
> > wfbm4_ets1_1
> > > down to
> > > real    0m1.118s
> > > user    0m1.640s
> > > sys     0m0.368s
> > > on my 1.66 GHz dual core laptop.
> >
> >
> > And on the 8-core 2.33GHz Intel Xeon Linux box with
> > 2 GB RAM, this version
> > is extremely fast:
> >
> > real 0m0.567s
> > user 0m2.249s
> > sys 0m0.956s
>
> (I'll ignore the unexplained sys time below. That
> makes the discussion a bit preliminary; perhaps the
> derived results should be computed some other way.
> Apply grain of salt appropriately.)
>
> For those keeping track, the latest result is fully
> 2.7 times faster than the best previous version (which
> was block read), and 17.3 times faster than the
> initial version. The latest speedup is basically due
> to doing less work.

Are we doing less work?
Not really, over 3 versions wfinder, wfinder1 and wfinder1_1
the only differences are how we do binary matching
- do NOT create sub binaries
- keep "pointers" for offsets inside one big binary
The reason that it get so much faster seems to be that
we have much less garbage collection.
wfinder
real    0m1.995s
user    0m3.164s
sys     0m0.396s
garbage collections:  46301
words reclaimed:   501744350

wfinder1
real    0m1.786s
user    0m2.792s
sys     0m0.364s
garbage collections:  36102
words reclaimed:  392187156


wfinder1_1
real    0m1.219s
user    0m1.660s
sys     0m0.376s
garbage collections: 10729
words reclaimed:  114930430

So the amount of garbage has been reduced by a factor > 4

>However, note that user time fell
> by a somewhat greater ratio than real time, which
> might mean parallelization overheads are becoming
> visible.
>
> Also, the user time of 2.249 seconds is now close to
> the Ruby user time, which were 2.095s on the same
> hardware, while the Erlang parallelization speedup
> (user/real) on top of this is 3.95 out of 8. Comparing
> the real times of Erlang (0.567s) and Ruby (2.21s), we
> get about the same execution time speedup, 3.9. Not
> too shabby, huh?
>
> Is there anything more to be wrung out of this
> program?

As I said yesterday, I think that the
current solution has a lower bound of ~0.5s no matter how
many cores You trow at it.

As Steve measured it
Workers  Real time
1              1.252
2              1.079
4              0.701
8              0.575
16            0.567

I think we have reached the limit on this track, we need to look at
this another way to make a solution, that probably is slower on
a low number of cores but that scales better on more > 4 cores.

>Well, apart from further tuning, one can note
> that Ruby had 0.1s sys time, while Erlang apparently
> needs a bit more, 0.5-1.0s. Why?
>
> Taking a wider view, it would also be very interesting
> to see how to apply these lessons to more general
> problems and/or libraries.

Some random thoughts
-The compiler should treat don't care variable (_Var) the same
as (_), and not bind them.
I like to be able to write
<<_Type:TLen/binary,_X:XLen/binary, Val:Len/binary....>>
instead of
SkipLen=TLen+XLen,
<<_:SkipLen/binary,Val:Len/binary...>>

- When doing folds over large binaries do not repetedly
split the binary in head and tail parts for recursive calls.
Keep an offset counter to track the position in the binary.

- One reason that it is even possible to solve the wide finder
in parallel is that it is possible to "resync" at a random place
in the file by locating a newline.
 I have an interest in processing files with BER coded data, so
I have been trying to make a BER version of widefinder, but since
it is necessary to scan the data sequentially to identify each
(TLV) block, it does not lend itself to the type of solution we
have for widefinder. My BER version currently takes
real    0m7.378s
user    0m6.768s
sys     0m0.876s

/Anders


From rpettit@REDACTED  Mon Oct 29 16:17:04 2007
From: rpettit@REDACTED (Rick Pettit)
Date: Mon, 29 Oct 2007 10:17:04 -0500
Subject: [erlang-questions] Erlang implementations for deployment
In-Reply-To: <de422a40710290737h61436c47s6ba5922ebd479294@mail.gmail.com>
References: <2D35C26B-76FF-47A7-A871-44B6F83F7D6F@geophile.com>
	<de422a40710290737h61436c47s6ba5922ebd479294@mail.gmail.com>
Message-ID: <20071029151704.GB31817@vailsys.com>

On Mon, Oct 29, 2007 at 03:37:07PM +0100, Kenneth Lundin wrote:
> Hi,
> 
> The term Commercial Erlang is out dated. There used to be be a
> "commercial" version with some extra components compared with the Open
> Source version but that was 10 years ago or so.
> Today there is only one version of Erlang OTP and that is what we
> deliver as Open Source Erlang.
> 
> When it comes to support we (Ericsson Erlang/OTP team) can
> unfortunately not deliver new support license contracts because of
> administrative reasons (has been like that several years now) but we
> are working on it and will hopefully in due time come to an
> arrangement that works.
> 
> Erlang on Linux is a good choice, Linux x86 is probably the most used
> platform for deployment of systems written in Erlang.
> 
> /Kenneth Erlang/OTP team at Ericsson
> 
> On 10/29/07, Jack Orenstein <jao@REDACTED> wrote:
> > Do production systems tend to be deployed using the open source
> > implementation of Erlang? Or is Ericsson's Commercial Erlang more
> > common? I'm particularly interested in running Erlang on Linux.

For what it's worth, I deploy OTP applications for routing telephony calls
(among many other uses) on solaris 8, solaris 9, and solaris 10 production
telephony systems requiring 24x7 uptime.

We have been using OTP in this capacity for over 5 years with much success.

-Rick


From mazen@REDACTED  Mon Oct 29 16:41:47 2007
From: mazen@REDACTED (Mazen Harake)
Date: Mon, 29 Oct 2007 15:41:47 +0000
Subject: [erlang-questions] :  idea: tagging subject line in this group
In-Reply-To: <4725D79D.2070709@ericsson.com>
References: <9b08084c0710261056u7bd59927ic71959d9a3e91758@mail.gmail.com>	<4e3934860710261110o4e8f8707v7ca714354652404b@mail.gmail.com>	<20071029085209.GB521@erix.ericsson.se>
	<4725D79D.2070709@ericsson.com>
Message-ID: <4725FF3B.4040808@erlang-consulting.com>

We should not forget:

offtopic:

or

ot:

;)

Ulf Wiger (TN/EAB) wrote:
> Raimo Niskanen wrote:
>   
>> The only problem has been to define the topics.
>> Can we take Joe's suggestion as it is?
>>     
>
> Joe's suggestions being:
>
>   
>>>> Can I suggest a simple tagging of the subject line. As follows:
>>>>
>>>>      idea:    An idea you want to share and get feedback on
>>>>      how:     A request for technical info
>>>>
>>>> Replies are just follow ups to the original posting.
>>>>
>>>>     erlounge: announce an erlounge
>>>>     beginner: a beginners question
>>>>     clarify:     please clarify ...
>>>>     program: I've written a fun program ...
>>>>     ad: an advertisement for something
>>>>     jobad: a job ad
>>>>     link: read this article it's interesting
>>>>         
>
> I think they're fine. I'd prefer 'job' instead of 'jobad',
> but that's just me.
>
> I think it's a very good idea.
>
> BR,
> Ulf W
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>   


-- 
Mazen Harake <mazen@REDACTED>
Erlang Software Developer and Consultant,
Erlang Training & Consulting, Ltd

Mobile Phone: +44 (0)795 13 26 317
Office Phone: +44 (0)207 45 61 020
Office Address:
401 London Fruit & Wool Exchange
Brushfield St, London, E1 6EL
United Kingdom

This email and its attachments may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of "Erlang Training & Consulting, Ltd".

If you are not the intended recipient of this email and its attachments, you must take no action based upon them, nor must you copy or show them to anyone. Please contact the sender if you believe you have received this email in error.



From anders.nygren@REDACTED  Mon Oct 29 16:31:56 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 09:31:56 -0600
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <003601c819ff$d1a1b5d0$74e52170$@com>
References: <4d08db370710280718p3484b8ddr1ce1481498e47769@mail.gmail.com>
	<20071028195217.GT6096@delora.autosys.us>
	<4d08db370710290019w67324875s206d4372efba0c89@mail.gmail.com>
	<003601c819ff$d1a1b5d0$74e52170$@com>
Message-ID: <d3520a2f0710290831k21eaa20aq179b6727a674c81a@mail.gmail.com>

On 10/29/07, Eranga Udesh <casper2000a@REDACTED> wrote:
> Hi,
>
> Is there a Open Source Sigtran stack developed preferably in Erlang? I mean
> actively developing and not a stale project. Anybody planning to do such
> thing?
>
> If there's none, I am interested to know how many of you would want somebody
> to make this initiative? If the interest level is high, I might be able to
> convince the relevant people in my office to Open Source the Sigtran stack
> we're developing. Let me know your interest.
>

Yes, we are interested. Assuming it is a license that allows commercial use.

/Anders

> BRgds,
> - Eranga
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bjorn@REDACTED  Mon Oct 29 16:54:29 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 29 Oct 2007 16:54:29 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
	<d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>
Message-ID: <o1odehvruy.fsf@erix.ericsson.se>

"Anders Nygren" <anders.nygren@REDACTED> writes:

> 
> Some random thoughts
> -The compiler should treat don't care variable (_Var) the same
> as (_), and not bind them.
> I like to be able to write
> <<_Type:TLen/binary,_X:XLen/binary, Val:Len/binary....>>
> instead of
> SkipLen=TLen+XLen,
> <<_:SkipLen/binary,Val:Len/binary...>>

The BEAM compiler treats all variables the same.

If the compiler finds that a variable is not used, it will NOT
match out the binary.

I assume that HiPE's native-code compiler does the same optimization.

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From anders.nygren@REDACTED  Mon Oct 29 17:24:17 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 10:24:17 -0600
Subject: [erlang-questions] widefinder update
In-Reply-To: <o1odehvruy.fsf@erix.ericsson.se>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
	<d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>
	<o1odehvruy.fsf@erix.ericsson.se>
Message-ID: <d3520a2f0710290924h5ffe4b23mfaed427d4e6c8184@mail.gmail.com>

On 29 Oct 2007 16:54:29 +0100, Bjorn Gustavsson <bjorn@REDACTED> wrote:
> "Anders Nygren" <anders.nygren@REDACTED> writes:
>
> >
> > Some random thoughts
> > -The compiler should treat don't care variable (_Var) the same
> > as (_), and not bind them.
> > I like to be able to write
> > <<_Type:TLen/binary,_X:XLen/binary, Val:Len/binary....>>
> > instead of
> > SkipLen=TLen+XLen,
> > <<_:SkipLen/binary,Val:Len/binary...>>
>
> The BEAM compiler treats all variables the same.
>
> If the compiler finds that a variable is not used, it will NOT
> match out the binary.

Good to hear that.

/Anders

>
> I assume that HiPE's native-code compiler does the same optimization.
>
> /Bjorn
> --
> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From puzza007@REDACTED  Mon Oct 29 16:39:03 2007
From: puzza007@REDACTED (Paul Oliver)
Date: Mon, 29 Oct 2007 15:39:03 +0000
Subject: [erlang-questions] port_command and marshalling
Message-ID: <a4cc77ec0710290839t4b47aceau7172a63f785205fe@mail.gmail.com>

Hi,

I have a port open with opts [{packet, 4}, binary] and call it using:

port_command(Port, term_to_binary(Msg)),

With the definition of  receive_erlang_port_msg below, for a
particular call, I get:

lengthstr[0]=0
lengthstr[1]=0
lengthstr[2]=5
lengthstr[3]=-103
LEN=-103

Does anyone know where the -103 comes from?

Is there any standard marshalling code I can lift?

Thanks!
Paul.

byte* receive_erlang_port_msg(void)
{
    int i, len = 0;
    byte *buffer;
    byte lengthstr[4];

    if (read_exact(lengthstr, 4) !=4) {
		exit(1);
    }

	for (i=0;i<4;i++) {
		fprintf(stderr,"lengthstr[%d]=%d\n",i, lengthstr[i]);
	}

	len = (unsigned) lengthstr[3];
	len |= ((unsigned) lengthstr[2]) << 8;
	len |= ((unsigned) lengthstr[1]) << 16;
	len |= ((unsigned) lengthstr[0]) << 24;

	fprintf(stderr, "LEN=%d\n", len);

    buffer = (byte *)malloc(len);

    if (read_exact(buffer, len) <= 0) {
		exit(1);
    }

    return buffer;
}


From per@REDACTED  Mon Oct 29 18:16:42 2007
From: per@REDACTED (Per Hedeland)
Date: Mon, 29 Oct 2007 18:16:42 +0100 (CET)
Subject: [erlang-questions] port_command and marshalling
In-Reply-To: <a4cc77ec0710290839t4b47aceau7172a63f785205fe@mail.gmail.com>
Message-ID: <200710291716.l9THGgmp063734@pluto.hedeland.org>

"Paul Oliver" <puzza007@REDACTED> wrotes:
>
>I have a port open with opts [{packet, 4}, binary] and call it using:
>
>port_command(Port, term_to_binary(Msg)),
>
>With the definition of  receive_erlang_port_msg below, for a
>particular call, I get:
>
>lengthstr[0]=0
>lengthstr[1]=0
>lengthstr[2]=5
>lengthstr[3]=-103
>LEN=-103
>
>Does anyone know where the -103 comes from?

It comes from 153 + your use of an apparently-signed type 'byte' for the
length bytes. If you switch that to 'unsigned char' it will work (at
least upto len = 2^31-1). Or cast the bytes to 'unsigned char' instead
of 'unsigned' (which isn't some "generic" thing, but just short for
'unsigned int').

--Per Hedeland


From valentin@REDACTED  Mon Oct 29 16:28:02 2007
From: valentin@REDACTED (Valentin Micic)
Date: Mon, 29 Oct 2007 17:28:02 +0200
Subject: [erlang-questions] EUC 2007 oversubscribed
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com><992686.15039.qm@web38809.mail.mud.yahoo.com><d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com><4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com><006a01c81986$1a359390$e751d854@Dell><f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com><811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com><4724DB0A.9000903@diit.unict.it>
	<eb8fc1930710290624iae32ab8x156d4fd216466d83@mail.gmail.com>
Message-ID: <015001c81a40$729e8680$6401a8c0@moneymaker2>

How about South Africa? You could enjoy nice weather, food & wine too, but 
you can get mugged any time of the day, as an added bonus -- surely nothing 
can beat that!

V.

----- Original Message ----- 
From: "dda" <headspin@REDACTED>
To: <erlang-questions@REDACTED>
Sent: Monday, October 29, 2007 3:24 PM
Subject: Re: [erlang-questions] EUC 2007 oversubscribed


> Well if weather is a concern, you wimps ;-), Hong Kong is an
> appropriate venue then! It's 26? C today, at 21:22. There *are*
> appropriate conference facilities, good food, and proselytising
> outside the Western world would do Erlang some good!
>
> -- 
> Didier
>
> On 10/29/07, Corrado Santoro <csanto@REDACTED> wrote:
>> Michael Campbell wrote:
>> > There are some really nice convention halls in Atlanta GA and Orlando 
>> > FL...  =)
>>
>> ... but the weather in Sicily, in November, is still very nice...
>> (Francesco knows)... and we have very huge halls too ;-)
>>
>> --Corrado
>>
>> --
>> ==================================================================
>> Eng. Corrado Santoro, Ph.D.
>> University of Catania - ITALY - Engineering Faculty
>>
>> Tel: +39 095 7382380        VoIP: sip:7035@REDACTED
>>
>> Personal Home Page: http://www.diit.unict.it/users/csanto
>>      NUXI Home Page: http://nuxi.diit.unict.it
>> ==================================================================
>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://www.erlang.org/mailman/listinfo/erlang-questions
>>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions 



From dmercer@REDACTED  Mon Oct 29 18:26:05 2007
From: dmercer@REDACTED (David Mercer)
Date: Mon, 29 Oct 2007 12:26:05 -0500
Subject: [erlang-questions] beginner: Updating Data Structures
Message-ID: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>

While an Erlang system has the ability to update its program on the fly,
updating data structures on the fly seems a bit more difficult.  Unless you
can upgrade all nodes simultaneously, some nodes will be expecting the old
data structure while others then new.  My question therefore, is how to
structure my data?  Is there an approach that I am missing that is both
upgrade-friendly and ETS/Mnesia-compatible?  Please see the following
paragraphs for my analysis so far.

 

Suppose we are writing an inventory control application.  We decide to
create a record to contain our information about items in our inventory.
Not much to say about items, really, so we?re just going to hold the item?s
name in a record.  If something else ever needs to be tracked regarding
these items, we can always upgrade our data, right?

 

-record(item,

       { name

       }).

 

So we roll out our new inventory system to 3,000 nodes in our 25 warehouses
in 6 different countries, and everything works swimmingly.  For a while.

 

However, some time later, our accounting department decides we need a way to
value our inventory, and each item should have a value associated with it.
That way, we can calculate inventory value simply by multiplying value by
quantity at each location.  Unfortunately, we cannot now use our record
structure.  What to do?

 

Well, na?vely, we decide to just modify our item record.

 

-record(item,

       { name

       , value

       }).

 

This new record structure is incompatible with the old item record
structure, so we will also write some code that upgrades our items in the
system to the new structure when we upgrade the system.  Unfortunately,
unless our entire worldwide operation is upgraded all at once, any process
using the old structure will crash when it encounters a new-style item, and
vice versa.  Simultaneous upgrading all 3,000 nodes is impractical, so we?ll
have to rethink our original decision.

 

We could have created the original record structure with expansion slots
available for future use.

 

-record(item,

       { name

       , 'X1'

       , 'X2'

       , 'X3'

       }).

 

Now when Accounting wants us to add the value of the item to the item
record, we simply redefine one of the expansion slots.

 

-record(item,

       { name

       , value

       , 'X2'

       , 'X3'

       }).

 

This will not crash any process, since the size of resulting tuple is still
the same.  Unfortunately, we might run out of expansion slots if we don?t
allocate enough of them.  The example runs out of slots once Accounting also
gets their cost-basis and GL-class elements added, leaving us in the same
boat as before.  We simply delayed the inevitable.  We might get bright and
allocate the new slots hierarchically by department, for instance, so
Accounting gets only one slot for all of its information, and we define a
new record for the information in that slot.

 

-record(item,

       { name

       , acctg

       , 'X2'

       , 'X3'

       }).

-record(acctg_item,

       { value

       , cost_basis

       , gl_class

       , 'X1'

       , 'X2'

       , 'X3'

       }).

 

However, this approach once again only delays the inevitable.  When
Inventory Control and Manufacturing take up the other two expansion slots,
there is no room for Engineering?s data.  Plus, we have multiplied this
problem, since it occurs for each of our subrecords, which can also run out
of expansion slots.

 

Another alternative might be to have only one expansion slot, which is
filled in by the next version of the item record.

 

-record(item,

       { name

       , v2

       }).

-record(item_v2,

       { value

       , cost_basis

       , gl_class

       , v3

       }).

 

Now when we have more elements to add, we create an item_v3 record (with a
v4 element to accommodate future expansion), and so on.  The problems with
this, however, are that programmers need to know which version of the record
a certain data element is, and that by the time we go through a few score
enhancements and we?re up to version 68, it becomes quite cumbersome, and is
little better than had we used a linked list.

 

In fact, a linked list may well be better.  Instead of writing functions
with the record syntax, we can use lists.

 

item_value([item, _Name, Value | _])

       ->

                Value

              .

 

To retrieve the value, we only need to know its position in the list.  This
approach suffers from a couple of problems: (1) You need to know the
position of each element in the list; (2) This list will be repeated quite
frequently, so when you have 300 attributes your code will be brittle,
repetitive, and difficult to maintain.

 

Perhaps an alternative approach is to define each record version
independently, instead of additively as we tried earlier.

 

-record(item1,

       { name

       }).

-record(item2,

       { item

       , value

       , cost_basis

       , gl_class

       }).

 

Now in our code, we have versions of each function matching on the record
structure, and a function that handles the no-match case (in case you?re
running v2 code when you receive a v3 record).  Once again, however, we run
into a couple of obstacles: (1) We must implement a different version of
each function for each version of the record (this will get tiresome around
version 68); (2) new versions are not backward compatible: a node running a
previous version of the code will not recognize future-versioned data
structures, even though the only fields it needs are those from its own
version.

 

Let?s borrow a page from object-oriented design principles.  Why not let the
item provide its own methods for data access through functions contained on
the structure.  We define a record ?class? which has two slots: one for the
methods, and one for the data.  By doing this, items carry around their own
methods and so it doesn?t really matter what version of an item something
is, so long as the item knows how to use its own data.  First we define some
infrastructure.

 

-module(class).

-export([invoke/3]).

-record(class,

       { methods

       , data

       }).

invoke(Method_ID, Object = #class{methods = Methods}, Args)

       ->

                Method = Methods(Method_ID),

                Method(Object, Args)

              .

 

To call a method on an object, syntax is simply ?invoke(Method_ID, Object,
Args)?, such as

 

X = item:new ("X"),  % Create a new item "X"

X_Name = class:invoke(get_name, X, []),  % Returns "X"

Y = class:invoke(set_name, X, ["Y"]).  % Changes item name

 

This is great for encapsulation!  The implementation is straightforward.

 

-module(item).

-export([new/1]).

-include("class.hrl").

-record(item,

       { name

       }).

 

new(Name)

       ->

                #class{ methods = fun(get_name) -> fun get_name/2

                                  ;  (set_name) -> fun set_name/2

                                  end

                      , data    = #item{ name = Name }

                      }

              .

              

get_name(#class{data = #item{name = Name}}, _)

       ->

                Name

              .

 

set_name(Object = #class{data = Item}, [Name])

       ->

                Object#class{data = Item#item{name = Name}}

              .

 

Alas, there is a fly in this ointment, too.  While it would appear that the
method functions are being carried around along with the data (in fact, the
item tuple is ?{class,#Fun<item.0.96410792>,{item,"X"}}?), those functions
are really not carried around from node to node.  Instead, Erlang only
carries around references to the functions.  This means if this item shows
up on a node where the function does not exist, an error will occur when a
method is invoked.

 

The fact that you cannot safely sling functions around with your data from
node to node indicates that perhaps we need a very simple interface with
functions that will never change.  Maybe instead of using records at all, we
can use basic OTP library functions to associate item properties with their
values.  Sounds kind of like what proplists were designed for.

 

X = [{name, "X"}],  % Create a new item "X"

X_Name = proplists:get_value(name, X),  % Returns "X"

Y = [{name, "Y"} | proplists:delete(name, X1)].  % Changes item name

 

A similar effect can be had with dicts, with the decision probably to be
made based on performance.  (Not only that, but the decision can be made
dynamically at run-time, since there are functions for converting between
the two.)

 

X = dict:from_list([{name, "X"}]),  % Create a new item "X"

X_Name = dict:fetch(name, X),  % Returns "X"

Y = dict:store(name, "Y", X).  % Changes item name

 

This approach has the advantage of being completely backward-compatible with
respect to my code-base.  Should a later version of our inventory
application add a property, it will not change the operation of any previous
version.  Once again, however, there are problems with this approach: (1)
property values cannot be used for matching in function definitions; (2)
these structures are not easily indexed: ETS and Mnesia require record data
types.  While Disadvantage 1 might be easily managed by performing lookups
and conditionals within the function, Disadvantage 2 is probably
intractable.

 

To repeat my question, gentle readers, how ought I structure my data?  Is
there an approach that I am missing that is both upgrade-friendly and
ETS/Mnesia-compatible?

 

Thank-you.

 

Cheers,

 

David

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/d8b7aee4/attachment.htm>

From kenneth.lundin@REDACTED  Mon Oct 29 17:40:36 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Mon, 29 Oct 2007 17:40:36 +0100
Subject: [erlang-questions] Erlang implementations for deployment
In-Reply-To: <20071029151704.GB31817@vailsys.com>
References: <2D35C26B-76FF-47A7-A871-44B6F83F7D6F@geophile.com>
	<de422a40710290737h61436c47s6ba5922ebd479294@mail.gmail.com>
	<20071029151704.GB31817@vailsys.com>
Message-ID: <de422a40710290940p22c3e152geeba3143fbd9631d@mail.gmail.com>

> For what it's worth, I deploy OTP applications for routing telephony calls
> (among many other uses) on solaris 8, solaris 9, and solaris 10 production
> telephony systems requiring 24x7 uptime.
>
> We have been using OTP in this capacity for over 5 years with much success.
>
> -Rick
>

Yes I should have written that Solaris and Linux are the most popular platforms
for Erlang based products as far as I know.

/Kenneth Erlang/OTP team at Ericsson


From matthias@REDACTED  Mon Oct 29 18:02:36 2007
From: matthias@REDACTED (Matthias Lang)
Date: Mon, 29 Oct 2007 18:02:36 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710290924h5ffe4b23mfaed427d4e6c8184@mail.gmail.com>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
	<d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>
	<o1odehvruy.fsf@erix.ericsson.se>
	<d3520a2f0710290924h5ffe4b23mfaed427d4e6c8184@mail.gmail.com>
Message-ID: <18214.4652.334366.44240@cors.corelatus.se>

Anders Nygren writes:

 > > > -The compiler should treat don't care variable (_Var) the same
 > > > as (_), and not bind them.

Careful here, it's possible to interpret what you wrote in ways which
lead to incorrect reasoning about programs.

_Var is not the "don't care" variable in general. As far as observable
behaviour goes, _Var *is* bound and the compiler is not allowed to not
bind variables of that form in general. Some examples where the
difference is observable:

  46> f(), <<_A, _B>> = <<1,2>>.
  <<1,2>>
  47> f(), <<_A, _A>> = <<1,2>>. 
  ** exited: {{badmatch,<<1,2>>},[{erl_eval,expr,3}]} **
  48> f(), <<_, _>> = <<1,2>>.  
  <<1,2>>

Variables longer than one character starting with an underscore are
special in only one way: they suppress a warning.

Naturally the compiler is free to optimise away variables which aren't
used.

Matthias


From anders.nygren@REDACTED  Mon Oct 29 19:54:38 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 12:54:38 -0600
Subject: [erlang-questions] A more parallel widefinder
Message-ID: <d3520a2f0710291154i2c88872crd268c9c58d55f99@mail.gmail.com>

One more try.
This one tries to do more in the workers and as little as possible in
the central controller.

The controller only
- starts workers, and maintains a specific number of workers.
- prints the results

Each worker
- opens the file and reads a chunk
- finds the first and last newlines in the chunk
- sends the partial line at the beginning of the chunk
to the worker processing the previous chunk
- processes its chunk, excluding the initial and final
partial lines
- receives a partial line from from the worker that handles
the next chunk
- concatenates and process the last line

The sequential control part now takes ~180 ms
A worker processing a 200kbytes chunk takes ~1.7 ms
So it will scale better to more cores than the previous versions.

I estimate that on Steve's 8 core it will run at ~ 0.3 s.

It is actually slightly faster than my previous wfinder1_1 even
on a dual core.

/Anders
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfinder3.erl
Type: text/x-erlang
Size: 4218 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/e6c9acdf/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wfbm4_ets2.erl
Type: text/x-erlang
Size: 3520 bytes
Desc: not available
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071029/e6c9acdf/attachment-0001.bin>

From sean.hinde@REDACTED  Mon Oct 29 20:24:49 2007
From: sean.hinde@REDACTED (Sean Hinde)
Date: Mon, 29 Oct 2007 19:24:49 +0000
Subject: [erlang-questions] beginner: Updating Data Structures
In-Reply-To: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
References: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
Message-ID: <CE39A4EF-AAF9-4458-B6AD-365D64E024FA@gmail.com>

Hi,

Perhaps you could consider a two stage upgrade. First upgrade all  
software to a version that understands the new record as well as the  
old (dynamically dispatching/converting on record size). Then once  
that is done invoke the command to tell nodes to start using the new  
record (perhaps also doing a few table transforms along the way).

It can sometimes help to use the fact that records are also tagged  
tuples. Kind of ugly in the code, but could be isolated to a small  
number of places.

Another option we have with some success is to have a single extension  
field in the record that holds a tagged tuple list. It is  
extraordinary how much such a structure can be abused ;-)

Sean

On 29 Oct 2007, at 17:26, David Mercer wrote:

> While an Erlang system has the ability to update its program on the  
> fly, updating data structures on the fly seems a bit more  
> difficult.  Unless you can upgrade all nodes simultaneously, some  
> nodes will be expecting the old data structure while others then  
> new.  My question therefore, is how to structure my data?  Is there  
> an approach that I am missing that is both upgrade-friendly and ETS/ 
> Mnesia-compatible?  Please see the following paragraphs for my  
> analysis so far.
>
> Suppose we are writing an inventory control application.  We decide  
> to create a record to contain our information about items in our  
> inventory.  Not much to say about items, really, so we?re just going  
> to hold the item?s name in a record.  If something else ever needs  
> to be tracked regarding these items, we can always upgrade our data,  
> right?
>
> -record(item,
>        { name
>        }).
>
> So we roll out our new inventory system to 3,000 nodes in our 25  
> warehouses in 6 different countries, and everything works  
> swimmingly.  For a while.
>
> However, some time later, our accounting department decides we need  
> a way to value our inventory, and each item should have a value  
> associated with it.  That way, we can calculate inventory value  
> simply by multiplying value by quantity at each location.   
> Unfortunately, we cannot now use our record structure.  What to do?
>
> Well, na?vely, we decide to just modify our item record.
>
> -record(item,
>        { name
>        , value
>        }).
>
> This new record structure is incompatible with the old item record  
> structure, so we will also write some code that upgrades our items  
> in the system to the new structure when we upgrade the system.   
> Unfortunately, unless our entire worldwide operation is upgraded all  
> at once, any process using the old structure will crash when it  
> encounters a new-style item, and vice versa.  Simultaneous upgrading  
> all 3,000 nodes is impractical, so we?ll have to rethink our  
> original decision.
>
> We could have created the original record structure with expansion  
> slots available for future use.
>
> -record(item,
>        { name
>        , 'X1'
>        , 'X2'
>        , 'X3'
>        }).
>
> Now when Accounting wants us to add the value of the item to the  
> item record, we simply redefine one of the expansion slots.
>
> -record(item,
>        { name
>        , value
>        , 'X2'
>        , 'X3'
>        }).
>
> This will not crash any process, since the size of resulting tuple  
> is still the same.  Unfortunately, we might run out of expansion  
> slots if we don?t allocate enough of them.  The example runs out of  
> slots once Accounting also gets their cost-basis and GL-class  
> elements added, leaving us in the same boat as before.  We simply  
> delayed the inevitable.  We might get bright and allocate the new  
> slots hierarchically by department, for instance, so Accounting gets  
> only one slot for all of its information, and we define a new record  
> for the information in that slot.
>
> -record(item,
>        { name
>        , acctg
>        , 'X2'
>        , 'X3'
>        }).
> -record(acctg_item,
>        { value
>        , cost_basis
>        , gl_class
>        , 'X1'
>        , 'X2'
>        , 'X3'
>        }).
>
> However, this approach once again only delays the inevitable.  When  
> Inventory Control and Manufacturing take up the other two expansion  
> slots, there is no room for Engineering?s data.  Plus, we have  
> multiplied this problem, since it occurs for each of our subrecords,  
> which can also run out of expansion slots.
>
> Another alternative might be to have only one expansion slot, which  
> is filled in by the next version of the item record.
>
> -record(item,
>        { name
>        , v2
>        }).
> -record(item_v2,
>        { value
>        , cost_basis
>        , gl_class
>        , v3
>        }).
>
> Now when we have more elements to add, we create an item_v3 record  
> (with a v4 element to accommodate future expansion), and so on.  The  
> problems with this, however, are that programmers need to know which  
> version of the record a certain data element is, and that by the  
> time we go through a few score enhancements and we?re up to version  
> 68, it becomes quite cumbersome, and is little better than had we  
> used a linked list.
>
> In fact, a linked list may well be better.  Instead of writing  
> functions with the record syntax, we can use lists.
>
> item_value([item, _Name, Value | _])
>        ->
>                 Value
>               .
>
> To retrieve the value, we only need to know its position in the  
> list.  This approach suffers from a couple of problems: (1) You need  
> to know the position of each element in the list; (2) This list will  
> be repeated quite frequently, so when you have 300 attributes your  
> code will be brittle, repetitive, and difficult to maintain.
>
> Perhaps an alternative approach is to define each record version  
> independently, instead of additively as we tried earlier.
>
> -record(item1,
>        { name
>        }).
> -record(item2,
>        { item
>        , value
>        , cost_basis
>        , gl_class
>        }).
>
> Now in our code, we have versions of each function matching on the  
> record structure, and a function that handles the no-match case (in  
> case you?re running v2 code when you receive a v3 record).  Once  
> again, however, we run into a couple of obstacles: (1) We must  
> implement a different version of each function for each version of  
> the record (this will get tiresome around version 68); (2) new  
> versions are not backward compatible: a node running a previous  
> version of the code will not recognize future-versioned data  
> structures, even though the only fields it needs are those from its  
> own version.
>
> Let?s borrow a page from object-oriented design principles.  Why not  
> let the item provide its own methods for data access through  
> functions contained on the structure.  We define a record ?class?  
> which has two slots: one for the methods, and one for the data.  By  
> doing this, items carry around their own methods and so it doesn?t  
> really matter what version of an item something is, so long as the  
> item knows how to use its own data.  First we define some  
> infrastructure.
>
> -module(class).
> -export([invoke/3]).
> -record(class,
>        { methods
>        , data
>        }).
> invoke(Method_ID, Object = #class{methods = Methods}, Args)
>        ->
>                 Method = Methods(Method_ID),
>                 Method(Object, Args)
>               .
>
> To call a method on an object, syntax is simply ?invoke(Method_ID,  
> Object, Args)?, such as
>
> X = item:new ("X"),  % Create a new item "X"
> X_Name = class:invoke(get_name, X, []),  % Returns "X"
> Y = class:invoke(set_name, X, ["Y"]).  % Changes item name
>
> This is great for encapsulation!  The implementation is  
> straightforward.
>
> -module(item).
> -export([new/1]).
> -include("class.hrl").
> -record(item,
>        { name
>        }).
>
> new(Name)
>        ->
>                 #class{ methods = fun(get_name) -> fun get_name/2
>                                   ;  (set_name) -> fun set_name/2
>                                   end
>                       , data    = #item{ name = Name }
>                       }
>               .
>
> get_name(#class{data = #item{name = Name}}, _)
>        ->
>                 Name
>               .
>
> set_name(Object = #class{data = Item}, [Name])
>        ->
>                 Object#class{data = Item#item{name = Name}}
>               .
>
> Alas, there is a fly in this ointment, too.  While it would appear  
> that the method functions are being carried around along with the  
> data (in fact, the item tuple is ?{class,#Fun<item.0.96410792>, 
> {item,"X"}}?), those functions are really not carried around from  
> node to node.  Instead, Erlang only carries around references to the  
> functions.  This means if this item shows up on a node where the  
> function does not exist, an error will occur when a method is invoked.
>
> The fact that you cannot safely sling functions around with your  
> data from node to node indicates that perhaps we need a very simple  
> interface with functions that will never change.  Maybe instead of  
> using records at all, we can use basic OTP library functions to  
> associate item properties with their values.  Sounds kind of like  
> what proplists were designed for.
>
> X = [{name, "X"}],  % Create a new item "X"
> X_Name = proplists:get_value(name, X),  % Returns "X"
> Y = [{name, "Y"} | proplists:delete(name, X1)].  % Changes item name
>
> A similar effect can be had with dicts, with the decision probably  
> to be made based on performance.  (Not only that, but the decision  
> can be made dynamically at run-time, since there are functions for  
> converting between the two.)
>
> X = dict:from_list([{name, "X"}]),  % Create a new item "X"
> X_Name = dict:fetch(name, X),  % Returns "X"
> Y = dict:store(name, "Y", X).  % Changes item name
>
> This approach has the advantage of being completely backward- 
> compatible with respect to my code-base.  Should a later version of  
> our inventory application add a property, it will not change the  
> operation of any previous version.  Once again, however, there are  
> problems with this approach: (1) property values cannot be used for  
> matching in function definitions; (2) these structures are not  
> easily indexed: ETS and Mnesia require record data types.  While  
> Disadvantage 1 might be easily managed by performing lookups and  
> conditionals within the function, Disadvantage 2 is probably  
> intractable.
>
> To repeat my question, gentle readers, how ought I structure my  
> data?  Is there an approach that I am missing that is both upgrade- 
> friendly and ETS/Mnesia-compatible?
>
> Thank-you.
>
> Cheers,
>
> David
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From puzza007@REDACTED  Mon Oct 29 21:37:43 2007
From: puzza007@REDACTED (Paul Oliver)
Date: Mon, 29 Oct 2007 20:37:43 +0000
Subject: [erlang-questions] port_command and marshalling
In-Reply-To: <200710291716.l9THGgmp063734@pluto.hedeland.org>
References: <a4cc77ec0710290839t4b47aceau7172a63f785205fe@mail.gmail.com>
	<200710291716.l9THGgmp063734@pluto.hedeland.org>
Message-ID: <a4cc77ec0710291337ucd171d0w84a962956382306a@mail.gmail.com>

Thanks Per!  That did the trick.

On 10/29/07, Per Hedeland <per@REDACTED> wrote:
> "Paul Oliver" <puzza007@REDACTED> wrotes:
> >
> >I have a port open with opts [{packet, 4}, binary] and call it using:
> >
> >port_command(Port, term_to_binary(Msg)),
> >
> >With the definition of  receive_erlang_port_msg below, for a
> >particular call, I get:
> >
> >lengthstr[0]=0
> >lengthstr[1]=0
> >lengthstr[2]=5
> >lengthstr[3]=-103
> >LEN=-103
> >
> >Does anyone know where the -103 comes from?
>
> It comes from 153 + your use of an apparently-signed type 'byte' for the
> length bytes. If you switch that to 'unsigned char' it will work (at
> least upto len = 2^31-1). Or cast the bytes to 'unsigned char' instead
> of 'unsigned' (which isn't some "generic" thing, but just short for
> 'unsigned int').
>
> --Per Hedeland
>


-- 
http://mosangeles.net


From paul-trapexit@REDACTED  Mon Oct 29 22:18:56 2007
From: paul-trapexit@REDACTED (Paul Mineiro)
Date: Mon, 29 Oct 2007 14:18:56 -0700 (PDT)
Subject: [erlang-questions] beginner: mnesia secondary indices with
	ordered_set
Message-ID: <Pine.LNX.4.52L2.0710291418360.15501@herbie>

hi.

when i use ordered_set mnesia table type, i expect the primary key to be
sorted and thus prefix lookups to be efficient.

is the same true of secondary indices on ordered_set tables?  i.e., can
i expect prefix lookups on secondary indices to be efficient?

thanks,

-- p


Optimism is an essential ingredient of innovation. How else can the
individual favor change over security?

  -- Robert Noyce


From erlang@REDACTED  Mon Oct 29 22:43:35 2007
From: erlang@REDACTED (Joe Armstrong)
Date: Mon, 29 Oct 2007 22:43:35 +0100
Subject: [erlang-questions] video: announce
Message-ID: <9b08084c0710291443x4d61eb82k6233098f684260be@mail.gmail.com>

http://channel9.msdn.com/ShowPost.aspx?PostID=351659#351659

:-)

/Joe


From bbmaj7@REDACTED  Mon Oct 29 23:48:30 2007
From: bbmaj7@REDACTED (Richard Andrews)
Date: Tue, 30 Oct 2007 09:48:30 +1100 (EST)
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <d3520a2f0710290831k21eaa20aq179b6727a674c81a@mail.gmail.com>
Message-ID: <460103.74419.qm@web52010.mail.re2.yahoo.com>


--- Anders Nygren <anders.nygren@REDACTED> wrote:

> > Is there a Open Source Sigtran stack developed preferably in Erlang? I mean
> > actively developing and not a stale project. Anybody planning to do such
> > thing?
> >
> > If there's none, I am interested to know how many of you would want
> somebody
> > to make this initiative? If the interest level is high, I might be able to
> > convince the relevant people in my office to Open Source the Sigtran stack
> > we're developing. Let me know your interest.
> >
> 
> Yes, we are interested. Assuming it is a license that allows commercial use.


It wouldn't be open source if it did not [1].

--

[1] http://www.opensource.org/docs/definition.php




      Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage.
http://au.docs.yahoo.com/mail/unlimitedstorage.html



From anders.nygren@REDACTED  Tue Oct 30 01:09:46 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Mon, 29 Oct 2007 18:09:46 -0600
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <460103.74419.qm@web52010.mail.re2.yahoo.com>
References: <d3520a2f0710290831k21eaa20aq179b6727a674c81a@mail.gmail.com>
	<460103.74419.qm@web52010.mail.re2.yahoo.com>
Message-ID: <d3520a2f0710291709k775aa433t48fa9cfc261f8a41@mail.gmail.com>

On 10/29/07, Richard Andrews <bbmaj7@REDACTED> wrote:
>
> --- Anders Nygren <anders.nygren@REDACTED> wrote:
>
> > > Is there a Open Source Sigtran stack developed preferably in Erlang? I mean
> > > actively developing and not a stale project. Anybody planning to do such
> > > thing?
> > >
> > > If there's none, I am interested to know how many of you would want
> > somebody
> > > to make this initiative? If the interest level is high, I might be able to
> > > convince the relevant people in my office to Open Source the Sigtran stack
> > > we're developing. Let me know your interest.
> > >
> >
> > Yes, we are interested. Assuming it is a license that allows commercial use.
>
>
> It wouldn't be open source if it did not [1].
>
I do not want to get into a license war but are You trying to tell me that
-LGPL
-BSD
-MIT
-Apache
etc etc

are NOT open source licenses?
see http://www.opensource.org/licenses/alphabetical

/Anders

> --
>
> [1] http://www.opensource.org/docs/definition.php
>


From fig@REDACTED  Tue Oct 30 02:31:24 2007
From: fig@REDACTED (Michael FIG)
Date: Mon, 29 Oct 2007 19:31:24 -0600 (CST)
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <d3520a2f0710291709k775aa433t48fa9cfc261f8a41@mail.gmail.com>
Message-ID: <16566264.150061193707884488.JavaMail.root@zimbra>

"Commercial" is an awful term to use when discussing software.

1) Open Source/Free Software explicitly requires that it is possible to sell the software product, so long as the source code is available, modifiable, and redistributable.  So, in that sense, it "allows commercial use."

2) However, many Free Software licenses don't allow a downstream developer to use the software in a traditional proprietary package (or any use that doesn't meet the requirements above).  So, in that sense, it "doesn't allow commercial use."

What I understand is that Anders wants #1, and Richard is saying he will offer #2.  They are not in conflict, if I am correct.

I hope this helps clarify things,

-- 
Michael FIG <fig@REDACTED>, PMP
MarkeTel Multi-Line Dialing Systems, Ltd.
Phone: (306) 359-6893 ext. 528

----- Original Message -----
From: "Anders Nygren" <anders.nygren@REDACTED>
To: "Richard Andrews" <bbmaj7@REDACTED>
Cc: erlang-questions@REDACTED
Sent: Monday, October 29, 2007 6:09:46 PM (GMT-0600) America/Guatemala
Subject: Re: [erlang-questions] Open Source ErlSigtran

On 10/29/07, Richard Andrews <bbmaj7@REDACTED> wrote:
>
> --- Anders Nygren <anders.nygren@REDACTED> wrote:
>
> > > Is there a Open Source Sigtran stack developed preferably in Erlang? I mean
> > > actively developing and not a stale project. Anybody planning to do such
> > > thing?
> > >
> > > If there's none, I am interested to know how many of you would want
> > somebody
> > > to make this initiative? If the interest level is high, I might be able to
> > > convince the relevant people in my office to Open Source the Sigtran stack
> > > we're developing. Let me know your interest.
> > >
> >
> > Yes, we are interested. Assuming it is a license that allows commercial use.
>
>
> It wouldn't be open source if it did not [1].
>
I do not want to get into a license war but are You trying to tell me that
-LGPL
-BSD
-MIT
-Apache
etc etc

are NOT open source licenses?
see http://www.opensource.org/licenses/alphabetical

/Anders

> --
>
> [1] http://www.opensource.org/docs/definition.php
>
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions



From vances@REDACTED  Tue Oct 30 02:55:20 2007
From: vances@REDACTED (Vance Shipley)
Date: Mon, 29 Oct 2007 21:55:20 -0400
Subject: [erlang-questions] widefinder update
In-Reply-To: <18214.4652.334366.44240@cors.corelatus.se>
References: <65b2728e0710282156j1e22290eyebf411c876ac884d@mail.gmail.com>
	<233934.61038.qm@web38810.mail.mud.yahoo.com>
	<d3520a2f0710290825w69e301a0nff84ce1ef538e7f0@mail.gmail.com>
	<o1odehvruy.fsf@erix.ericsson.se>
	<d3520a2f0710290924h5ffe4b23mfaed427d4e6c8184@mail.gmail.com>
	<18214.4652.334366.44240@cors.corelatus.se>
Message-ID: <20071030015439.GA1848@little-black-book.motivity.ca>

On Mon, Oct 29, 2007 at 06:02:36PM +0100, Matthias Lang wrote:
}  Variables longer than one character starting with an underscore are
}  special in only one way: they suppress a warning.

Which was a mistake in my mind but it's probably a bikeshed
issue so ... never mind.

	-Vance


From rsaccon@REDACTED  Tue Oct 30 05:26:50 2007
From: rsaccon@REDACTED (Roberto Saccon)
Date: Tue, 30 Oct 2007 01:26:50 -0300
Subject: [erlang-questions] anybody using textmate as Erlang editor ?
In-Reply-To: <4724C8E3.8030903@gmail.com>
References: <eb4177b00710270029t2f838982ma40c5f4db058a028@mail.gmail.com>
	<4724C8E3.8030903@gmail.com>
Message-ID: <eb4177b00710292126obd5c1bdsedba73fba6b848be@mail.gmail.com>

Mateusz, thanks. I just wrote about my first experience with
textmate/erlang here:
http://www.rsaccon.com/2007/10/erlang-and-textmate.html

On 10/28/07, Mateusz Berezecki <mateuszb@REDACTED> wrote:
> Roberto Saccon wrote:
> > this is a very nice mac text editor, with Erlang syntax highlighting.
> > Is anybody using this for Erlang coding and maybe even tried to  adapt
> > it a bit more to Erlang, a la Emacs (if that is possible at all) ?
> >
> > regards
> >
> Hello Roberto,
>
> I am regularly using textmate for all of my programming work
> and with erlang there is no exceptions. So yes, people use it for Erlang
> programming :)
> FWIW I have not tried playing with grammars, syntax, etc. for better
> erlang support.
> The current state of erlang support in textmate is pretty good for me.
>
>
> Mateusz Berezecki
>


-- 
Roberto Saccon
http://rsaccon.com


From casper2000a@REDACTED  Tue Oct 30 09:28:59 2007
From: casper2000a@REDACTED (Eranga Udesh)
Date: Tue, 30 Oct 2007 13:58:59 +0530
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <16566264.150061193707884488.JavaMail.root@zimbra>
References: <d3520a2f0710291709k775aa433t48fa9cfc261f8a41@mail.gmail.com>
	<16566264.150061193707884488.JavaMail.root@zimbra>
Message-ID: <004b01c81ace$eb895410$c29bfc30$@com>

You are quite right. To my understanding, 

GPL like licenses wants the ISV or VARs, who uses the GPL licensed product
to distribute their own product under GPL. Otherwise there's provision to
buy commercial license. License cost is most of the time a fraction of a
fully commercialized product.

Also there's Apache/Apple like licenses which allow to distribute your
product commercially/closed source, while the Open Source product must be
distribute in source code version with the license attached. 

Both model are highly successful. Under 1st model there're project driving
companies, hence faster development and support, while in the 2nd case it's
most of the time like communities.

- Eranga



-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of Michael FIG
Sent: Tuesday, October 30, 2007 7:01 AM
To: Anders Nygren
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] Open Source ErlSigtran

"Commercial" is an awful term to use when discussing software.

1) Open Source/Free Software explicitly requires that it is possible to sell
the software product, so long as the source code is available, modifiable,
and redistributable.  So, in that sense, it "allows commercial use."

2) However, many Free Software licenses don't allow a downstream developer
to use the software in a traditional proprietary package (or any use that
doesn't meet the requirements above).  So, in that sense, it "doesn't allow
commercial use."

What I understand is that Anders wants #1, and Richard is saying he will
offer #2.  They are not in conflict, if I am correct.

I hope this helps clarify things,

-- 
Michael FIG <fig@REDACTED>, PMP
MarkeTel Multi-Line Dialing Systems, Ltd.
Phone: (306) 359-6893 ext. 528

----- Original Message -----
From: "Anders Nygren" <anders.nygren@REDACTED>
To: "Richard Andrews" <bbmaj7@REDACTED>
Cc: erlang-questions@REDACTED
Sent: Monday, October 29, 2007 6:09:46 PM (GMT-0600) America/Guatemala
Subject: Re: [erlang-questions] Open Source ErlSigtran

On 10/29/07, Richard Andrews <bbmaj7@REDACTED> wrote:
>
> --- Anders Nygren <anders.nygren@REDACTED> wrote:
>
> > > Is there a Open Source Sigtran stack developed preferably in Erlang? I
mean
> > > actively developing and not a stale project. Anybody planning to do
such
> > > thing?
> > >
> > > If there's none, I am interested to know how many of you would want
> > somebody
> > > to make this initiative? If the interest level is high, I might be
able to
> > > convince the relevant people in my office to Open Source the Sigtran
stack
> > > we're developing. Let me know your interest.
> > >
> >
> > Yes, we are interested. Assuming it is a license that allows commercial
use.
>
>
> It wouldn't be open source if it did not [1].
>
I do not want to get into a license war but are You trying to tell me that
-LGPL
-BSD
-MIT
-Apache
etc etc

are NOT open source licenses?
see http://www.opensource.org/licenses/alphabetical

/Anders

> --
>
> [1] http://www.opensource.org/docs/definition.php
>
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions




From vychodil.hynek@REDACTED  Tue Oct 30 10:25:41 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Tue, 30 Oct 2007 10:25:41 +0100
Subject: [erlang-questions] A more parallel widefinder
In-Reply-To: <d3520a2f0710291154i2c88872crd268c9c58d55f99@mail.gmail.com>
References: <d3520a2f0710291154i2c88872crd268c9c58d55f99@mail.gmail.com>
Message-ID: <4d08db370710300225y631a7c4q699f4d50fbcfff59@mail.gmail.com>

Good work. It's very nice idea to find last new line in chunk and than
not concatenate binary with big prefix. But I still argue, that this
implementation is not so much like universal tool, but especial one
purpose tuned solution. I'm so busy to try rewrite it to more modular
implementation. But I will try it some time.

Congratulation anyway.

--Hynek (Pichi) Vychodil

On 10/29/07, Anders Nygren <anders.nygren@REDACTED> wrote:
> One more try.
> This one tries to do more in the workers and as little as possible in
> the central controller.
>
> The controller only
> - starts workers, and maintains a specific number of workers.
> - prints the results
>
> Each worker
> - opens the file and reads a chunk
> - finds the first and last newlines in the chunk
> - sends the partial line at the beginning of the chunk
> to the worker processing the previous chunk
> - processes its chunk, excluding the initial and final
> partial lines
> - receives a partial line from from the worker that handles
> the next chunk
> - concatenates and process the last line
>
> The sequential control part now takes ~180 ms
> A worker processing a 200kbytes chunk takes ~1.7 ms
> So it will scale better to more cores than the previous versions.
>
> I estimate that on Steve's 8 core it will run at ~ 0.3 s.
>
> It is actually slightly faster than my previous wfinder1_1 even
> on a dual core.
>
> /Anders
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
>


From monch1962@REDACTED  Tue Oct 30 11:01:31 2007
From: monch1962@REDACTED (David Mitchell)
Date: Tue, 30 Oct 2007 21:01:31 +1100
Subject: [erlang-questions] Instant Erlang?
Message-ID: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>

Hello all,

I'm working in an environment where I'd like to be able to implement a
number of Erlang systems on existing Windows servers.  Furthermore, I
need to be able to bring them up quickly, and remove them without
leaving a trace.

Something along the lines of InstantRails
(http://instantrails.rubyforge.org/wiki/wiki.pl), but for Erlang,
would be ideal.  "Installing" InstantRails doesn't touch the Windows
registry at all - instead, you get a way of bringing up a command
shell that includes the path to the Rails "stuff", and you can drive
Rails from within that command shell without actually making any
registry changes.

A dig through Google doesn't give me any clues, so then I hit on the
idea of installing eJabberd for Windows.  This goes close - it gives a
simple install process, a fully-fledged Erlang config (plus eJabberd,
which I can ignore), plus a good uninstall process - but it's probably
going to take some persuasion of the environment owners to let me
install it because it makes a bunch of registry entries.

Do any of you know of an Instant Rails-like, drop-in/drop-out, Erlang
installation that already exists?  I could possibly look into building
one myself, but realistically there's no enough hours in the day for
me to be able to implement it in the near term.

Alternately, any other suggestions along the lines of eJabberd - a
solution that goes close to zero install, but has a few caveats -
would also be welcome?

Thanks in advance for any suggestions

Dave M.


From christophe.romain@REDACTED  Tue Oct 30 11:50:46 2007
From: christophe.romain@REDACTED (Christophe Romain)
Date: Tue, 30 Oct 2007 11:50:46 +0100
Subject: [erlang-questions] Instant Erlang?
In-Reply-To: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
References: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
Message-ID: <4C3EE81A-1A8F-452E-8930-D206721F6DED@process-one.net>

ejabberd installer changes the registry to allow starting ejabberd as  
e service. It is based on CEAN archives.
CEAN do not use registry at all. Did you tryed CEAN archive for  
windows ?
it's a one click erlang deployment.

http://cean.process-one.net/
http://cean.process-one.net/R11B/prod/windows/cean_installer.exe



From monch1962@REDACTED  Tue Oct 30 12:05:07 2007
From: monch1962@REDACTED (David Mitchell)
Date: Tue, 30 Oct 2007 22:05:07 +1100
Subject: [erlang-questions] Interfacing Erlang With IBM's Websphere MQ
In-Reply-To: <e9447a400710231623h136852afv1c3803ca1a2c5ea2@mail.gmail.com>
References: <006101c814c7$d702c6f0$891ea8c0@SSI.CORP>
	<e9447a400710221123lf324babod94b4209ac67f797@mail.gmail.com>
	<00da01c815bb$44a4fc70$891ea8c0@SSI.CORP>
	<e9447a400710231623h136852afv1c3803ca1a2c5ea2@mail.gmail.com>
Message-ID: <f6508a860710300405i3d70633cj93b72e60935c2cac@mail.gmail.com>

Looking for a solution to the same problem - if you can post any code
for such a Java or C port, I'd really appreciate if you did so.

Thanks in advance

Dave M.

On 24/10/2007, Patrick Logan <patrickdlogan@REDACTED> wrote:
> > > Heavier mechanisms for Erlang would be one of the several Java
> > > interfaces (JMS or MQ-specific), then go through Erlang's Java
> > > library.
> > >
> > > Or the MQ C/C++ interfaces.
> >
> > Is that the same as developing a Java or C/C++ port, or is this something
> > different?
>
> You'd have to wrap whichever approach up into a C or Java port.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From ulf@REDACTED  Tue Oct 30 11:54:21 2007
From: ulf@REDACTED (Ulf Wiger)
Date: Tue, 30 Oct 2007 11:54:21 +0100
Subject: [erlang-questions] beginner: mnesia secondary indices with
	ordered_set
In-Reply-To: <Pine.LNX.4.52L2.0710291418360.15501@herbie>
References: <Pine.LNX.4.52L2.0710291418360.15501@herbie>
Message-ID: <8209f740710300354r3578ebfbt9a72a5c84dd8d955@mail.gmail.com>

No, Mnesia has only one type of index.
The rdbms contrib in Jungerl implements index tables a bit
differently (as full mnesia tables), and supports ordered index,
among other things.

But to the best of my knowledge, rdbms is not actively used
by anyone (unless Rudolph van Graan is still using it).

BR,
Ulf W

2007/10/29, Paul Mineiro <paul-trapexit@REDACTED>:
> hi.
>
> when i use ordered_set mnesia table type, i expect the primary key to be
> sorted and thus prefix lookups to be efficient.
>
> is the same true of secondary indices on ordered_set tables?  i.e., can
> i expect prefix lookups on secondary indices to be efficient?
>
> thanks,
>
> -- p
>
>
> Optimism is an essential ingredient of innovation. How else can the
> individual favor change over security?
>
>   -- Robert Noyce
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From david.hopwood@REDACTED  Mon Oct 29 01:26:09 2007
From: david.hopwood@REDACTED (David Hopwood)
Date: Mon, 29 Oct 2007 00:26:09 +0000
Subject: [erlang-questions] widefinder update
In-Reply-To: <d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
Message-ID: <472528A1.5090005@industrial-designers.co.uk>

Anders Nygren wrote:
> On 10/28/07, Thomas Lindgren <thomasl_erlang@REDACTED> wrote:
>
>> In this particular case, you could increase I/O
>> performance by, say, striping the disk. And you can
>> increase CPU performance by, say, distributing the
>> work to multiple hosts/nodes (fairly straightforward
>> with Erlang, by the way). But with these problems,
>> even with infinite hardware you will eventually run
>> into some sequential portion of the code, and that
>> will limit the speedup as per Amdahl's Law.
> 
> Currently that sequential part is ~ 0.5s on my 1.66GHz
> dual core laptop. the part of the work that can be run in parallel
> takes ~2.254 s so theoretically we would get
> Cores  Real time  Speedup  Rel. speedup by doubling #cores
[...]
>   8      0.782     3.523      1.360
[...]
> 256      0.509     5.413      1.017
> 
> Which is not very good after 8 cores.

0.5 s is not very long, in human terms. For me to be convinced that
there is any need for further optimization, the problem would have
to be scaled to a point where the total run-time is something that
a human might conceivably get impatient waiting for. At that point,
the sequential part would likely be a smaller proportion of the
run-time anyway.

-- 
David Hopwood <david.hopwood@REDACTED>



From erlang@REDACTED  Tue Oct 30 12:17:05 2007
From: erlang@REDACTED (Hubert Matthews)
Date: Tue, 30 Oct 2007 11:17:05 +0000
Subject: [erlang-questions] Instant Erlang?
In-Reply-To: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
References: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
Message-ID: <472712B1.4030702@oxyware.com>

There is no need to touch the Windows registry to have a program run. 
  The usual reason for registry entries is to enable the "Add/Remove 
Programs" functionality through the Control Panel.  All you need to do 
is to put all of your stuff in a directory and have all of the 
executables and DLLs in a single directory.  That way you don't even 
have to change the PATH envvar.  I haven't done this for Erlang but 
this is what I do for my native Windows apps.

-- 
Hubert Matthews         http://www.oxyware.com/
Software Consultant     hubert@REDACTED



From peter@REDACTED  Tue Oct 30 13:27:29 2007
From: peter@REDACTED (Peter K Chan)
Date: Tue, 30 Oct 2007 13:27:29 +0100
Subject: [erlang-questions] Instant Erlang?
In-Reply-To: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
Message-ID: <A250ACDE71BB1847AB72F6D84DB4F3B533B7A0@ntxbeus24.exchange.xchg>

David,
	Search on SAE (Standalone Erlang) for some past discussion for
client-side erlang discussion option.

	In particular, when I brought up a similar question in the past
about finding a deployment setup, someone suggested looking into
Wings3D, which is also erlang based. I took a look at it and it seems to
have a very clean installation structure, which you can easily customize
for your own application.

Peter

-----Original Message-----
From: erlang-questions-bounces@REDACTED
[mailto:erlang-questions-bounces@REDACTED] On Behalf Of David Mitchell
Sent: Tuesday, October 30, 2007 5:02 AM
To: Erlang
Subject: [erlang-questions] Instant Erlang?

Hello all,

I'm working in an environment where I'd like to be able to implement a
number of Erlang systems on existing Windows servers.  Furthermore, I
need to be able to bring them up quickly, and remove them without
leaving a trace.

Something along the lines of InstantRails
(http://instantrails.rubyforge.org/wiki/wiki.pl), but for Erlang,
would be ideal.  "Installing" InstantRails doesn't touch the Windows
registry at all - instead, you get a way of bringing up a command
shell that includes the path to the Rails "stuff", and you can drive
Rails from within that command shell without actually making any
registry changes.

A dig through Google doesn't give me any clues, so then I hit on the
idea of installing eJabberd for Windows.  This goes close - it gives a
simple install process, a fully-fledged Erlang config (plus eJabberd,
which I can ignore), plus a good uninstall process - but it's probably
going to take some persuasion of the environment owners to let me
install it because it makes a bunch of registry entries.

Do any of you know of an Instant Rails-like, drop-in/drop-out, Erlang
installation that already exists?  I could possibly look into building
one myself, but realistically there's no enough hours in the day for
me to be able to implement it in the near term.

Alternately, any other suggestions along the lines of eJabberd - a
solution that goes close to zero install, but has a few caveats -
would also be welcome?

Thanks in advance for any suggestions

Dave M.
_______________________________________________
erlang-questions mailing list
erlang-questions@REDACTED
http://www.erlang.org/mailman/listinfo/erlang-questions


From vychodil.hynek@REDACTED  Tue Oct 30 13:39:45 2007
From: vychodil.hynek@REDACTED (Hynek Vychodil)
Date: Tue, 30 Oct 2007 13:39:45 +0100
Subject: [erlang-questions] widefinder update
In-Reply-To: <472528A1.5090005@industrial-designers.co.uk>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<472528A1.5090005@industrial-designers.co.uk>
Message-ID: <4d08db370710300539v22418899i5fb69ade7b0bd9b0@mail.gmail.com>

On 10/29/07, David Hopwood <david.hopwood@REDACTED> wrote:
> Anders Nygren wrote:
> >
> > Currently that sequential part is ~ 0.5s on my 1.66GHz
> > dual core laptop. the part of the work that can be run in parallel
> > takes ~2.254 s so theoretically we would get
> > Cores  Real time  Speedup  Rel. speedup by doubling #cores
> [...]
> >   8      0.782     3.523      1.360
> [...]
> > 256      0.509     5.413      1.017
> >
> > Which is not very good after 8 cores.
>
> 0.5 s is not very long, in human terms. For me to be convinced that
> there is any need for further optimization, the problem would have
> to be scaled to a point where the total run-time is something that
> a human might conceivably get impatient waiting for. At that point,
> the sequential part would likely be a smaller proportion of the
> run-time anyway.
>

It's bad point of view. What if you would like do it 100 times? 50s is
still good time in human terms? And what about 20GB instead 200MB? The
serial part will be 50s, it's wrong in human terms again. If you can
change serial part three times down, you made three times faster
solution. That's all. And what about web service? You can use three
times cheaper HW to serve same amount of users. It's not enough? Tim
Brays exercise is not only 200MB ~ 1Mrec, but also 1GB ~ 5Mrec, but in
this case 2.5s is long time in human terms if it will may be Web/UI
response. 1.7s will be better anyway.

--Hynek (Pichi) Vychodil


From mogorman@REDACTED  Tue Oct 30 13:57:01 2007
From: mogorman@REDACTED (Matthew O'Gorman)
Date: Tue, 30 Oct 2007 07:57:01 -0500
Subject: [erlang-questions] Open Source ErlSigtran
In-Reply-To: <004b01c81ace$eb895410$c29bfc30$@com>
References: <d3520a2f0710291709k775aa433t48fa9cfc261f8a41@mail.gmail.com>
	<16566264.150061193707884488.JavaMail.root@zimbra>
	<004b01c81ace$eb895410$c29bfc30$@com>
Message-ID: <f49fd40d0710300557gfd78aebp91affb4402d93724@mail.gmail.com>

You are incorrect.  As copyright holder you can release code under
multiple licenses, for example mysql, they control of the copyright
for the project and thus release it under the gpl as well as a
commercial license, when someone contributes to them they require the
patch be given the copyright.   The only advantage of an apple like
license is it states some of this up front, where as an apache license
allows anyone to roll it into a closed program.   I tend to think the
gpl allows for the maximum amount of freedom for the users and if you
want to sell propitiatory licenses its easy if you are the copyright
holder.

Mog


From anders.nygren@REDACTED  Tue Oct 30 16:58:54 2007
From: anders.nygren@REDACTED (Anders Nygren)
Date: Tue, 30 Oct 2007 09:58:54 -0600
Subject: [erlang-questions] A more parallel widefinder
In-Reply-To: <4d08db370710300225y631a7c4q699f4d50fbcfff59@mail.gmail.com>
References: <d3520a2f0710291154i2c88872crd268c9c58d55f99@mail.gmail.com>
	<4d08db370710300225y631a7c4q699f4d50fbcfff59@mail.gmail.com>
Message-ID: <d3520a2f0710300858w6601fccevbb1b891e7cd33cce@mail.gmail.com>

On 10/30/07, Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
> Good work. It's very nice idea to find last new line in chunk and than
> not concatenate binary with big prefix. But I still argue, that this
> implementation is not so much like universal tool, but especial one
> purpose tuned solution. I'm so busy to try rewrite it to more modular
> implementation. But I will try it some time.

Hynek
I think the idea can have universal use, for the class of data where
it is possible to start scanning at an arbitrary position and identify
the start or end of a logical data unit.

Scanning the entire file to identify the block separators creates a large
sequential term that can not take advantage to more cores.

/Anders


From bjorn@REDACTED  Tue Oct 30 18:50:57 2007
From: bjorn@REDACTED (Bjorn Gustavsson)
Date: 30 Oct 2007 18:50:57 +0100
Subject: [erlang-questions] A more parallel widefinder
In-Reply-To: <d3520a2f0710300858w6601fccevbb1b891e7cd33cce@mail.gmail.com>
References: <d3520a2f0710291154i2c88872crd268c9c58d55f99@mail.gmail.com>
	<4d08db370710300225y631a7c4q699f4d50fbcfff59@mail.gmail.com>
	<d3520a2f0710300858w6601fccevbb1b891e7cd33cce@mail.gmail.com>
Message-ID: <o11wbc4hku.fsf@erix.ericsson.se>

"Anders Nygren" <anders.nygren@REDACTED> writes:

> On 10/30/07, Hynek Vychodil <vychodil.hynek@REDACTED> wrote:
> > Good work. It's very nice idea to find last new line in chunk and than
> > not concatenate binary with big prefix. But I still argue, that this
> > implementation is not so much like universal tool, but especial one
> > purpose tuned solution. I'm so busy to try rewrite it to more modular
> > implementation. But I will try it some time.
> 
> Hynek
> I think the idea can have universal use, for the class of data where
> it is possible to start scanning at an arbitrary position and identify
> the start or end of a logical data unit.

I agree.

I think that this trick could be used to speed the crashdump viewer
(but probably not for R12B-0). A crash dump could be huge, and
even though the crashdump viewer doesn't read all information to memory
at once, it needs to read through the entire file once to index it
(to know, for instance, where information about each particular process
can be found in the crash dump file).

/Bjorn
-- 
Bj?rn Gustavsson, Erlang/OTP, Ericsson AB


From dot@REDACTED  Tue Oct 30 19:05:55 2007
From: dot@REDACTED (Tony Finch)
Date: Tue, 30 Oct 2007 18:05:55 +0000
Subject: [erlang-questions] R12 snapshots
In-Reply-To: <o11wbhggkt.fsf@erix.ericsson.se>
References: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
	<o11wbhggkt.fsf@erix.ericsson.se>
Message-ID: <Pine.LNX.4.64.0710301800250.23828@hermes-1.csi.cam.ac.uk>

What does the B in the Erlang release number indicate?

Tony.
-- 
f.a.n.finch  <dot@REDACTED>  http://dotat.at/
HUMBER THAMES: NORTHWEST BACKING SOUTHWEST 4 OR 5, OCCASIONALLY 6 LATER.
SLIGHT OR MODERATE. SHOWERS. GOOD.


From dmercer@REDACTED  Tue Oct 30 20:03:36 2007
From: dmercer@REDACTED (David Mercer)
Date: Tue, 30 Oct 2007 14:03:36 -0500
Subject: [erlang-questions] beginner: Updating Data Structures
In-Reply-To: <CE39A4EF-AAF9-4458-B6AD-365D64E024FA@gmail.com>
References: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
	<CE39A4EF-AAF9-4458-B6AD-365D64E024FA@gmail.com>
Message-ID: <000301c81b27$947a32d0$891ea8c0@SSI.CORP>

My understanding of how a two-stage upgrade would work follows.

Prior to the upgrade, we have the following record:

> -record(item1,
>        { name
>        }).

In Stage 1 of the upgrade, support for the following record structure is
added, but the data is not itself updated:

> -record(item2,
>        { item
>        , value
>        , cost_basis
>        , gl_class
>        }).

Our Stage 1 upgrade code must be written to support both item1?s and
item2?s, but it does not upgrade the data structures yet.  Only once all
3,000 nodes of our system have been upgraded to Stage 1 can we initiate
Stage 2: upgrade all our 'item1' data structures to 'item2'.  Optionally, we
can also purge all references to the 'item1' structure.

This approach seems problematic for the following reasons: (1) the time and
administrative overhead required to release a new version is doubled; (2)
you may run into the situation in which the stages take so long to complete
that we have multiple upgrades happening across the system at once; (3) code
has to be rewritten every release if it handles items (to accept both
'item1' and 'item2' structures), even if it is not directly affected by the
change.

For example, take the following scenario:

A new release is ordered which requires a change to the item record.  All
the code that deals with items (which is almost all of it) is duplicated and
changed to allow it to work with both the old item structure and the new.
When this is completed, a Stage 1 upgrade is ordered worldwide.  Frankfurt
and Singapore complete their Stage 1 upgrade quickly, but unfortunately our
Los Angeles operation is caught up in a legal requirement requiring them to
notify a particular client two weeks in advance of any system update.
Meanwhile, our data center in Maputo, Mozambique has been having unspecified
?problems? upgrading that is putting the whole worldwide release of these
new features on hold.  Even after the two-week North American hold is
lifted, Maputo still has not upgraded to Stage 1.

Meanwhile, while waiting to hear back from Maputo (which continues to
demur), Software has released a new version with yet a different version of
the item record.  Now Frankfurt and Singapore have code running that works
with all three formats, Los Angeles has two, and Maputo is still on its
first, and *still* no new functionality has been released.

Is this the best we can do?

Cheers,

David

-----Original Message-----
From: Sean Hinde [mailto:sean.hinde@REDACTED] 
Sent: Monday, October 29, 2007 14:25
To: dmercer@REDACTED
Cc: erlang-questions@REDACTED
Subject: Re: [erlang-questions] beginner: Updating Data Structures

Hi,

Perhaps you could consider a two stage upgrade. First upgrade all  
software to a version that understands the new record as well as the  
old (dynamically dispatching/converting on record size). Then once  
that is done invoke the command to tell nodes to start using the new  
record (perhaps also doing a few table transforms along the way).

It can sometimes help to use the fact that records are also tagged  
tuples. Kind of ugly in the code, but could be isolated to a small  
number of places.

Another option we have with some success is to have a single extension  
field in the record that holds a tagged tuple list. It is  
extraordinary how much such a structure can be abused ;-)

Sean

On 29 Oct 2007, at 17:26, David Mercer wrote:

> While an Erlang system has the ability to update its program on the  
> fly, updating data structures on the fly seems a bit more  
> difficult.  Unless you can upgrade all nodes simultaneously, some  
> nodes will be expecting the old data structure while others then  
> new.  My question therefore, is how to structure my data?  Is there  
> an approach that I am missing that is both upgrade-friendly and ETS/ 
> Mnesia-compatible?  Please see the following paragraphs for my  
> analysis so far.
>
> Suppose we are writing an inventory control application.  We decide  
> to create a record to contain our information about items in our  
> inventory.  Not much to say about items, really, so we?re just going  
> to hold the item?s name in a record.  If something else ever needs  
> to be tracked regarding these items, we can always upgrade our data,  
> right?
>
> -record(item,
>        { name
>        }).
>
> So we roll out our new inventory system to 3,000 nodes in our 25  
> warehouses in 6 different countries, and everything works  
> swimmingly.  For a while.
>
> However, some time later, our accounting department decides we need  
> a way to value our inventory, and each item should have a value  
> associated with it.  That way, we can calculate inventory value  
> simply by multiplying value by quantity at each location.   
> Unfortunately, we cannot now use our record structure.  What to do?
>
> Well, na?vely, we decide to just modify our item record.
>
> -record(item,
>        { name
>        , value
>        }).
>
> This new record structure is incompatible with the old item record  
> structure, so we will also write some code that upgrades our items  
> in the system to the new structure when we upgrade the system.   
> Unfortunately, unless our entire worldwide operation is upgraded all  
> at once, any process using the old structure will crash when it  
> encounters a new-style item, and vice versa.  Simultaneous upgrading  
> all 3,000 nodes is impractical, so we?ll have to rethink our  
> original decision.
>
> We could have created the original record structure with expansion  
> slots available for future use.
>
> -record(item,
>        { name
>        , 'X1'
>        , 'X2'
>        , 'X3'
>        }).
>
> Now when Accounting wants us to add the value of the item to the  
> item record, we simply redefine one of the expansion slots.
>
> -record(item,
>        { name
>        , value
>        , 'X2'
>        , 'X3'
>        }).
>
> This will not crash any process, since the size of resulting tuple  
> is still the same.  Unfortunately, we might run out of expansion  
> slots if we don?t allocate enough of them.  The example runs out of  
> slots once Accounting also gets their cost-basis and GL-class  
> elements added, leaving us in the same boat as before.  We simply  
> delayed the inevitable.  We might get bright and allocate the new  
> slots hierarchically by department, for instance, so Accounting gets  
> only one slot for all of its information, and we define a new record  
> for the information in that slot.
>
> -record(item,
>        { name
>        , acctg
>        , 'X2'
>        , 'X3'
>        }).
> -record(acctg_item,
>        { value
>        , cost_basis
>        , gl_class
>        , 'X1'
>        , 'X2'
>        , 'X3'
>        }).
>
> However, this approach once again only delays the inevitable.  When  
> Inventory Control and Manufacturing take up the other two expansion  
> slots, there is no room for Engineering?s data.  Plus, we have  
> multiplied this problem, since it occurs for each of our subrecords,  
> which can also run out of expansion slots.
>
> Another alternative might be to have only one expansion slot, which  
> is filled in by the next version of the item record.
>
> -record(item,
>        { name
>        , v2
>        }).
> -record(item_v2,
>        { value
>        , cost_basis
>        , gl_class
>        , v3
>        }).
>
> Now when we have more elements to add, we create an item_v3 record  
> (with a v4 element to accommodate future expansion), and so on.  The  
> problems with this, however, are that programmers need to know which  
> version of the record a certain data element is, and that by the  
> time we go through a few score enhancements and we?re up to version  
> 68, it becomes quite cumbersome, and is little better than had we  
> used a linked list.
>
> In fact, a linked list may well be better.  Instead of writing  
> functions with the record syntax, we can use lists.
>
> item_value([item, _Name, Value | _])
>        ->
>                 Value
>               .
>
> To retrieve the value, we only need to know its position in the  
> list.  This approach suffers from a couple of problems: (1) You need  
> to know the position of each element in the list; (2) This list will  
> be repeated quite frequently, so when you have 300 attributes your  
> code will be brittle, repetitive, and difficult to maintain.
>
> Perhaps an alternative approach is to define each record version  
> independently, instead of additively as we tried earlier.
>
> -record(item1,
>        { name
>        }).
> -record(item2,
>        { item
>        , value
>        , cost_basis
>        , gl_class
>        }).
>
> Now in our code, we have versions of each function matching on the  
> record structure, and a function that handles the no-match case (in  
> case you?re running v2 code when you receive a v3 record).  Once  
> again, however, we run into a couple of obstacles: (1) We must  
> implement a different version of each function for each version of  
> the record (this will get tiresome around version 68); (2) new  
> versions are not backward compatible: a node running a previous  
> version of the code will not recognize future-versioned data  
> structures, even though the only fields it needs are those from its  
> own version.
>
> Let?s borrow a page from object-oriented design principles.  Why not  
> let the item provide its own methods for data access through  
> functions contained on the structure.  We define a record ?class?  
> which has two slots: one for the methods, and one for the data.  By  
> doing this, items carry around their own methods and so it doesn?t  
> really matter what version of an item something is, so long as the  
> item knows how to use its own data.  First we define some  
> infrastructure.
>
> -module(class).
> -export([invoke/3]).
> -record(class,
>        { methods
>        , data
>        }).
> invoke(Method_ID, Object = #class{methods = Methods}, Args)
>        ->
>                 Method = Methods(Method_ID),
>                 Method(Object, Args)
>               .
>
> To call a method on an object, syntax is simply ?invoke(Method_ID,  
> Object, Args)?, such as
>
> X = item:new ("X"),  % Create a new item "X"
> X_Name = class:invoke(get_name, X, []),  % Returns "X"
> Y = class:invoke(set_name, X, ["Y"]).  % Changes item name
>
> This is great for encapsulation!  The implementation is  
> straightforward.
>
> -module(item).
> -export([new/1]).
> -include("class.hrl").
> -record(item,
>        { name
>        }).
>
> new(Name)
>        ->
>                 #class{ methods = fun(get_name) -> fun get_name/2
>                                   ;  (set_name) -> fun set_name/2
>                                   end
>                       , data    = #item{ name = Name }
>                       }
>               .
>
> get_name(#class{data = #item{name = Name}}, _)
>        ->
>                 Name
>               .
>
> set_name(Object = #class{data = Item}, [Name])
>        ->
>                 Object#class{data = Item#item{name = Name}}
>               .
>
> Alas, there is a fly in this ointment, too.  While it would appear  
> that the method functions are being carried around along with the  
> data (in fact, the item tuple is ?{class,#Fun<item.0.96410792>, 
> {item,"X"}}?), those functions are really not carried around from  
> node to node.  Instead, Erlang only carries around references to the  
> functions.  This means if this item shows up on a node where the  
> function does not exist, an error will occur when a method is invoked.
>
> The fact that you cannot safely sling functions around with your  
> data from node to node indicates that perhaps we need a very simple  
> interface with functions that will never change.  Maybe instead of  
> using records at all, we can use basic OTP library functions to  
> associate item properties with their values.  Sounds kind of like  
> what proplists were designed for.
>
> X = [{name, "X"}],  % Create a new item "X"
> X_Name = proplists:get_value(name, X),  % Returns "X"
> Y = [{name, "Y"} | proplists:delete(name, X1)].  % Changes item name
>
> A similar effect can be had with dicts, with the decision probably  
> to be made based on performance.  (Not only that, but the decision  
> can be made dynamically at run-time, since there are functions for  
> converting between the two.)
>
> X = dict:from_list([{name, "X"}]),  % Create a new item "X"
> X_Name = dict:fetch(name, X),  % Returns "X"
> Y = dict:store(name, "Y", X).  % Changes item name
>
> This approach has the advantage of being completely backward- 
> compatible with respect to my code-base.  Should a later version of  
> our inventory application add a property, it will not change the  
> operation of any previous version.  Once again, however, there are  
> problems with this approach: (1) property values cannot be used for  
> matching in function definitions; (2) these structures are not  
> easily indexed: ETS and Mnesia require record data types.  While  
> Disadvantage 1 might be easily managed by performing lookups and  
> conditionals within the function, Disadvantage 2 is probably  
> intractable.
>
> To repeat my question, gentle readers, how ought I structure my  
> data?  Is there an approach that I am missing that is both upgrade- 
> friendly and ETS/Mnesia-compatible?
>
> Thank-you.
>
> Cheers,
>
> David
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions



From yinso.chen@REDACTED  Tue Oct 30 20:38:22 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 30 Oct 2007 12:38:22 -0700
Subject: [erlang-questions] beginner: Updating Data Structures
In-Reply-To: <000301c81b27$947a32d0$891ea8c0@SSI.CORP>
References: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
	<CE39A4EF-AAF9-4458-B6AD-365D64E024FA@gmail.com>
	<000301c81b27$947a32d0$891ea8c0@SSI.CORP>
Message-ID: <779bf2730710301238y3ed847bbpca18932d98ab23fa@mail.gmail.com>

Perhaps a different way to think about the two-stage upgrade is to decouple
data definition from the database and the code?

Instead of using the record definition in both the persistence as well as
the code, you pass the data through a transformation function.  That way in
the database you have

-record(item2, {name, value, ...}).

But in the code, you have

-record(item1, {name, ...}).

In between, you have

load(DB) ->
  Tuple = read_from_database_structure(),
  convert_from_item2_to_item1(Tuple).

Thus - you can first upgrade your db (care is needed to deal with locking,
etc) in a script, and once that's done, you can upgrade code, which is in
general a faster operation.  Minimize dependencies IMO is effective to
isolate and deal with change.

Cheers,

On 10/30/07, David Mercer <dmercer@REDACTED> wrote:
>
> My understanding of how a two-stage upgrade would work follows.
>
> Prior to the upgrade, we have the following record:
>
> > -record(item1,
> >        { name
> >        }).
>
> In Stage 1 of the upgrade, support for the following record structure is
> added, but the data is not itself updated:
>
> > -record(item2,
> >        { item
> >        , value
> >        , cost_basis
> >        , gl_class
> >        }).
>
> Our Stage 1 upgrade code must be written to support both item1's and
> item2's, but it does not upgrade the data structures yet.  Only once all
> 3,000 nodes of our system have been upgraded to Stage 1 can we initiate
> Stage 2: upgrade all our 'item1' data structures to 'item2'.  Optionally,
> we
> can also purge all references to the 'item1' structure.
>
> This approach seems problematic for the following reasons: (1) the time
> and
> administrative overhead required to release a new version is doubled; (2)
> you may run into the situation in which the stages take so long to
> complete
> that we have multiple upgrades happening across the system at once; (3)
> code
> has to be rewritten every release if it handles items (to accept both
> 'item1' and 'item2' structures), even if it is not directly affected by
> the
> change.
>
> For example, take the following scenario:
>
> A new release is ordered which requires a change to the item record.  All
> the code that deals with items (which is almost all of it) is duplicated
> and
> changed to allow it to work with both the old item structure and the new.
> When this is completed, a Stage 1 upgrade is ordered worldwide.  Frankfurt
> and Singapore complete their Stage 1 upgrade quickly, but unfortunately
> our
> Los Angeles operation is caught up in a legal requirement requiring them
> to
> notify a particular client two weeks in advance of any system update.
> Meanwhile, our data center in Maputo, Mozambique has been having
> unspecified
> "problems" upgrading that is putting the whole worldwide release of these
> new features on hold.  Even after the two-week North American hold is
> lifted, Maputo still has not upgraded to Stage 1.
>
> Meanwhile, while waiting to hear back from Maputo (which continues to
> demur), Software has released a new version with yet a different version
> of
> the item record.  Now Frankfurt and Singapore have code running that works
> with all three formats, Los Angeles has two, and Maputo is still on its
> first, and *still* no new functionality has been released.
>
> Is this the best we can do?
>
> Cheers,
>
> David
>
> -----Original Message-----
> From: Sean Hinde [mailto:sean.hinde@REDACTED]
> Sent: Monday, October 29, 2007 14:25
> To: dmercer@REDACTED
> Cc: erlang-questions@REDACTED
> Subject: Re: [erlang-questions] beginner: Updating Data Structures
>
> Hi,
>
> Perhaps you could consider a two stage upgrade. First upgrade all
> software to a version that understands the new record as well as the
> old (dynamically dispatching/converting on record size). Then once
> that is done invoke the command to tell nodes to start using the new
> record (perhaps also doing a few table transforms along the way).
>
> It can sometimes help to use the fact that records are also tagged
> tuples. Kind of ugly in the code, but could be isolated to a small
> number of places.
>
> Another option we have with some success is to have a single extension
> field in the record that holds a tagged tuple list. It is
> extraordinary how much such a structure can be abused ;-)
>
> Sean
>
> On 29 Oct 2007, at 17:26, David Mercer wrote:
>
> > While an Erlang system has the ability to update its program on the
> > fly, updating data structures on the fly seems a bit more
> > difficult.  Unless you can upgrade all nodes simultaneously, some
> > nodes will be expecting the old data structure while others then
> > new.  My question therefore, is how to structure my data?  Is there
> > an approach that I am missing that is both upgrade-friendly and ETS/
> > Mnesia-compatible?  Please see the following paragraphs for my
> > analysis so far.
> >
> > Suppose we are writing an inventory control application.  We decide
> > to create a record to contain our information about items in our
> > inventory.  Not much to say about items, really, so we're just going
> > to hold the item's name in a record.  If something else ever needs
> > to be tracked regarding these items, we can always upgrade our data,
> > right?
> >
> > -record(item,
> >        { name
> >        }).
> >
> > So we roll out our new inventory system to 3,000 nodes in our 25
> > warehouses in 6 different countries, and everything works
> > swimmingly.  For a while.
> >
> > However, some time later, our accounting department decides we need
> > a way to value our inventory, and each item should have a value
> > associated with it.  That way, we can calculate inventory value
> > simply by multiplying value by quantity at each location.
> > Unfortunately, we cannot now use our record structure.  What to do?
> >
> > Well, na?vely, we decide to just modify our item record.
> >
> > -record(item,
> >        { name
> >        , value
> >        }).
> >
> > This new record structure is incompatible with the old item record
> > structure, so we will also write some code that upgrades our items
> > in the system to the new structure when we upgrade the system.
> > Unfortunately, unless our entire worldwide operation is upgraded all
> > at once, any process using the old structure will crash when it
> > encounters a new-style item, and vice versa.  Simultaneous upgrading
> > all 3,000 nodes is impractical, so we'll have to rethink our
> > original decision.
> >
> > We could have created the original record structure with expansion
> > slots available for future use.
> >
> > -record(item,
> >        { name
> >        , 'X1'
> >        , 'X2'
> >        , 'X3'
> >        }).
> >
> > Now when Accounting wants us to add the value of the item to the
> > item record, we simply redefine one of the expansion slots.
> >
> > -record(item,
> >        { name
> >        , value
> >        , 'X2'
> >        , 'X3'
> >        }).
> >
> > This will not crash any process, since the size of resulting tuple
> > is still the same.  Unfortunately, we might run out of expansion
> > slots if we don't allocate enough of them.  The example runs out of
> > slots once Accounting also gets their cost-basis and GL-class
> > elements added, leaving us in the same boat as before.  We simply
> > delayed the inevitable.  We might get bright and allocate the new
> > slots hierarchically by department, for instance, so Accounting gets
> > only one slot for all of its information, and we define a new record
> > for the information in that slot.
> >
> > -record(item,
> >        { name
> >        , acctg
> >        , 'X2'
> >        , 'X3'
> >        }).
> > -record(acctg_item,
> >        { value
> >        , cost_basis
> >        , gl_class
> >        , 'X1'
> >        , 'X2'
> >        , 'X3'
> >        }).
> >
> > However, this approach once again only delays the inevitable.  When
> > Inventory Control and Manufacturing take up the other two expansion
> > slots, there is no room for Engineering's data.  Plus, we have
> > multiplied this problem, since it occurs for each of our subrecords,
> > which can also run out of expansion slots.
> >
> > Another alternative might be to have only one expansion slot, which
> > is filled in by the next version of the item record.
> >
> > -record(item,
> >        { name
> >        , v2
> >        }).
> > -record(item_v2,
> >        { value
> >        , cost_basis
> >        , gl_class
> >        , v3
> >        }).
> >
> > Now when we have more elements to add, we create an item_v3 record
> > (with a v4 element to accommodate future expansion), and so on.  The
> > problems with this, however, are that programmers need to know which
> > version of the record a certain data element is, and that by the
> > time we go through a few score enhancements and we're up to version
> > 68, it becomes quite cumbersome, and is little better than had we
> > used a linked list.
> >
> > In fact, a linked list may well be better.  Instead of writing
> > functions with the record syntax, we can use lists.
> >
> > item_value([item, _Name, Value | _])
> >        ->
> >                 Value
> >               .
> >
> > To retrieve the value, we only need to know its position in the
> > list.  This approach suffers from a couple of problems: (1) You need
> > to know the position of each element in the list; (2) This list will
> > be repeated quite frequently, so when you have 300 attributes your
> > code will be brittle, repetitive, and difficult to maintain.
> >
> > Perhaps an alternative approach is to define each record version
> > independently, instead of additively as we tried earlier.
> >
> > -record(item1,
> >        { name
> >        }).
> > -record(item2,
> >        { item
> >        , value
> >        , cost_basis
> >        , gl_class
> >        }).
> >
> > Now in our code, we have versions of each function matching on the
> > record structure, and a function that handles the no-match case (in
> > case you're running v2 code when you receive a v3 record).  Once
> > again, however, we run into a couple of obstacles: (1) We must
> > implement a different version of each function for each version of
> > the record (this will get tiresome around version 68); (2) new
> > versions are not backward compatible: a node running a previous
> > version of the code will not recognize future-versioned data
> > structures, even though the only fields it needs are those from its
> > own version.
> >
> > Let's borrow a page from object-oriented design principles.  Why not
> > let the item provide its own methods for data access through
> > functions contained on the structure.  We define a record "class"
> > which has two slots: one for the methods, and one for the data.  By
> > doing this, items carry around their own methods and so it doesn't
> > really matter what version of an item something is, so long as the
> > item knows how to use its own data.  First we define some
> > infrastructure.
> >
> > -module(class).
> > -export([invoke/3]).
> > -record(class,
> >        { methods
> >        , data
> >        }).
> > invoke(Method_ID, Object = #class{methods = Methods}, Args)
> >        ->
> >                 Method = Methods(Method_ID),
> >                 Method(Object, Args)
> >               .
> >
> > To call a method on an object, syntax is simply "invoke(Method_ID,
> > Object, Args)", such as
> >
> > X = item:new ("X"),  % Create a new item "X"
> > X_Name = class:invoke(get_name, X, []),  % Returns "X"
> > Y = class:invoke(set_name, X, ["Y"]).  % Changes item name
> >
> > This is great for encapsulation!  The implementation is
> > straightforward.
> >
> > -module(item).
> > -export([new/1]).
> > -include("class.hrl").
> > -record(item,
> >        { name
> >        }).
> >
> > new(Name)
> >        ->
> >                 #class{ methods = fun(get_name) -> fun get_name/2
> >                                   ;  (set_name) -> fun set_name/2
> >                                   end
> >                       , data    = #item{ name = Name }
> >                       }
> >               .
> >
> > get_name(#class{data = #item{name = Name}}, _)
> >        ->
> >                 Name
> >               .
> >
> > set_name(Object = #class{data = Item}, [Name])
> >        ->
> >                 Object#class{data = Item#item{name = Name}}
> >               .
> >
> > Alas, there is a fly in this ointment, too.  While it would appear
> > that the method functions are being carried around along with the
> > data (in fact, the item tuple is "{class,#Fun<item.0.96410792>,
> > {item,"X"}}"), those functions are really not carried around from
> > node to node.  Instead, Erlang only carries around references to the
> > functions.  This means if this item shows up on a node where the
> > function does not exist, an error will occur when a method is invoked.
> >
> > The fact that you cannot safely sling functions around with your
> > data from node to node indicates that perhaps we need a very simple
> > interface with functions that will never change.  Maybe instead of
> > using records at all, we can use basic OTP library functions to
> > associate item properties with their values.  Sounds kind of like
> > what proplists were designed for.
> >
> > X = [{name, "X"}],  % Create a new item "X"
> > X_Name = proplists:get_value(name, X),  % Returns "X"
> > Y = [{name, "Y"} | proplists:delete(name, X1)].  % Changes item name
> >
> > A similar effect can be had with dicts, with the decision probably
> > to be made based on performance.  (Not only that, but the decision
> > can be made dynamically at run-time, since there are functions for
> > converting between the two.)
> >
> > X = dict:from_list([{name, "X"}]),  % Create a new item "X"
> > X_Name = dict:fetch(name, X),  % Returns "X"
> > Y = dict:store(name, "Y", X).  % Changes item name
> >
> > This approach has the advantage of being completely backward-
> > compatible with respect to my code-base.  Should a later version of
> > our inventory application add a property, it will not change the
> > operation of any previous version.  Once again, however, there are
> > problems with this approach: (1) property values cannot be used for
> > matching in function definitions; (2) these structures are not
> > easily indexed: ETS and Mnesia require record data types.  While
> > Disadvantage 1 might be easily managed by performing lookups and
> > conditionals within the function, Disadvantage 2 is probably
> > intractable.
> >
> > To repeat my question, gentle readers, how ought I structure my
> > data?  Is there an approach that I am missing that is both upgrade-
> > friendly and ETS/Mnesia-compatible?
> >
> > Thank-you.
> >
> > Cheers,
> >
> > David
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questions@REDACTED
> > http://www.erlang.org/mailman/listinfo/erlang-questions
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071030/42ba69a0/attachment.htm>

From monch1962@REDACTED  Tue Oct 30 20:41:09 2007
From: monch1962@REDACTED (David Mitchell)
Date: Wed, 31 Oct 2007 06:41:09 +1100
Subject: [erlang-questions] Instant Erlang?
In-Reply-To: <da7121a891ea0511b5eb6ae2b081d445@tombob.com>
References: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
	<da7121a891ea0511b5eb6ae2b081d445@tombob.com>
Message-ID: <f6508a860710301241g87541e1j3d75a0d975df52d9@mail.gmail.com>

Thanks everyone - plenty of viable options to try.

I'll do some experimenting with just ZIPping up the Erlang directory
and my app, and then just unzipping it on the target machine - if that
works, it'd be ideal.

Failing that, the CEAN option looks good - although it's not quite as
simple as an unzip, it's pretty much the next best solution.

Time to get to work...

Thanks and regards

Dave M.

On 31/10/2007, Robert Raschke <rrerlang@REDACTED> wrote:
> It's been a while since I did this, but if I remember correctly, you
> can pretty much just zip up an existing Erlang installation and then
> unzip to the same location on a different machine and it will work
> (minus anything you've changed to make it host dependent).
>
> If you want it in a different location, then you need to edit erl.ini
> in bin and erts/bin appropriately.
>
> I think only erlsrv uses the registry.
>
> Robby
>
>
>


From kenneth.lundin@REDACTED  Tue Oct 30 22:46:42 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Tue, 30 Oct 2007 22:46:42 +0100
Subject: [erlang-questions] R12 snapshots
In-Reply-To: <Pine.LNX.4.64.0710301800250.23828@hermes-1.csi.cam.ac.uk>
References: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
	<o11wbhggkt.fsf@erix.ericsson.se>
	<Pine.LNX.4.64.0710301800250.23828@hermes-1.csi.cam.ac.uk>
Message-ID: <de422a40710301446s246997das99beb7c01b77b394@mail.gmail.com>

Hi,

The B as used in R10B, R11B, R12B has historical reasons.
When the Erlang/OTP started as a product within Ericsson even before it
became Open Source we started with R1A as a kind of beta-release which
was then followed by R1B as the stable release. We then continued in
that manner with R2A as a beta release followed by R2B a few months
later.
The same procedure have then been repeated with a few exceptions with
for example R5A, R5B and R5C. Somewhere along the road we became so
confident regarding our quality that we skipped the A releases and
went directly to the B release and that's where we are now.

Today we have R11B as a major release and then R11B-0,R11B-1,
...R11B-5 as service releases.

Next major release will be R12B (coming real soon now)

/Kenneth Erlang/OTP team at Ericsson

On 10/30/07, Tony Finch <dot@REDACTED> wrote:
> What does the B in the Erlang release number indicate?
>
> Tony.
> --
> f.a.n.finch  <dot@REDACTED>  http://dotat.at/
> HUMBER THAMES: NORTHWEST BACKING SOUTHWEST 4 OR 5, OCCASIONALLY 6 LATER.
> SLIGHT OR MODERATE. SHOWERS. GOOD.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From nindeu@REDACTED  Tue Oct 30 23:14:48 2007
From: nindeu@REDACTED (nindeu@REDACTED)
Date: Tue, 30 Oct 2007 23:14:48 +0100
Subject: [erlang-questions] Restrict epmd to one IP address?
Message-ID: <20071030221448.67090@gmx.net>

Hi,

Just searched a while for an option to bind epmd to a specific IP address. On my system it binds to all available IP addresses. Hmm, nothing so far. Do I really have to install a firewall because of epmd?

How do you deal with epmd? Any better options?

Regards,
Eric
-- 
Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger


From ok@REDACTED  Tue Oct 30 23:17:46 2007
From: ok@REDACTED (Richard A. O'Keefe)
Date: Wed, 31 Oct 2007 11:17:46 +1300
Subject: [erlang-questions] beginner: Updating Data Structures
In-Reply-To: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
References: <005201c81a50$d56e9880$891ea8c0@SSI.CORP>
Message-ID: <8588150C-F01A-43BC-946F-F4954CAB7C9C@cs.otago.ac.nz>

On 30 Oct 2007, at 6:26 am, David Mercer wrote
[about the trouble involved in upgrading by adding a field to a record].

This is precisely the problem that Joe Armstrong and I have proposed
essentially the same solution to.  In particular,
if you have
	V = <item{name=>'fred', value=>27}>
then
	<item{name=>Name}> = V
should work and should bind Name = 'fred'.  This would require about
the same amount of space as current records; it would be a little slower
but not much; and it would not involve any preprocessor whatever.

The effect is that old code can extract information from new records
safely and correctly (as long as the meaning of the fields has not
changed) and can revise fields it knows about (copy this record with
these fields changed to those values); the only thing that doesn't work
is old code creating records from scratch which are read by new code.




From rrerlang@REDACTED  Tue Oct 30 14:23:24 2007
From: rrerlang@REDACTED (Robert Raschke)
Date: Tue, 30 Oct 2007 13:23:24 +0000
Subject: [erlang-questions] Instant Erlang?
In-Reply-To: <f6508a860710300301q1eebaaf5v637f8beb42b51b19@mail.gmail.com>
Message-ID: <8726009ede0e6716c8b0b4853fea8c2a@tombob.com>

It's been a while since I did this, but if I remember correctly, you
can pretty much just zip up an existing Erlang installation and then
unzip to the same location on a different machine and it will work
(minus anything you've changed to make it host dependent).

If you want it in a different location, then you need to edit erl.ini
in bin and erts/bin appropriately.

I think only erlsrv uses the registry.

Robby




From erlangx@REDACTED  Wed Oct 31 01:07:50 2007
From: erlangx@REDACTED (Michael McDaniel)
Date: Tue, 30 Oct 2007 17:07:50 -0700
Subject: [erlang-questions] how: Re:  Restrict epmd to one IP address?
In-Reply-To: <20071030221448.67090@gmx.net>
References: <20071030221448.67090@gmx.net>
Message-ID: <20071031000750.GS21717@delora.autosys.us>

On Tue, Oct 30, 2007 at 11:14:48PM +0100, nindeu@REDACTED wrote:
> Hi,
> 
> Just searched a while for an option to bind epmd to a specific IP address. On my system it binds to all available IP addresses. Hmm, nothing so far. Do I really have to install a firewall because of epmd?
> 
> How do you deal with epmd? Any better options?
> 
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 Mssrs. Google liked the following search request

  erlang min max epmd
 
  providing me http://thefactoryfactory.com/wordpress/?p=319 as the
  third link.

~Michael


> Regards,
> Eric
> -- 
> Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
> Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 
> !DSPAM:52,4727ad0b261057532028631!
> 
> 

-- 
Michael McDaniel
Portland, Oregon, USA
http://autosys.us
+1 503 283 5284


From bernied@REDACTED  Wed Oct 31 02:17:30 2007
From: bernied@REDACTED (Bernhard Damberger)
Date: Tue, 30 Oct 2007 18:17:30 -0700 (PDT)
Subject: [erlang-questions]  Erlang "jar" file?
Message-ID: <13501250.post@talk.nabble.com>


Does erlang support the idea of an archived collection of compiled code, like
a jar file? I know that Joe Armstrong wrote a tool called ear that creates
an archive for stand alone erlang (see http://www.sics.se/~joe/sae.html).
But I got the impression that this is not supported by the standard erlang
distribution (or is no longer being actively developed).

All the interpreter really needs to do is to be able to load the .beam files
in from an archived file (say zip format). So, can it currently do so?

_bernhard
-- 
View this message in context: http://www.nabble.com/Erlang-%22jar%22-file--tf4722329.html#a13501250
Sent from the Erlang Questions mailing list archive at Nabble.com.



From yinso.chen@REDACTED  Wed Oct 31 02:46:39 2007
From: yinso.chen@REDACTED (YC)
Date: Tue, 30 Oct 2007 18:46:39 -0700
Subject: [erlang-questions] how: string stream?
Message-ID: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>

Hi all -

are there something equivalent to a string stream in Erlang?  The io module
utilizes a IoDevice(), but it seems an IoDevice is either a file or socket,
and unclear how to extend it.

I'm also wondering how to implement the equivalent of chaining streams
together to perform layers of processing - any examples are appreciated.

Thanks,
yc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071030/5d975e29/attachment.htm>

From nindeu@REDACTED  Wed Oct 31 03:28:06 2007
From: nindeu@REDACTED (nindeu@REDACTED)
Date: Wed, 31 Oct 2007 03:28:06 +0100
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <20071031000750.GS21717@delora.autosys.us>
References: <20071030221448.67090@gmx.net>
	<20071031000750.GS21717@delora.autosys.us>
Message-ID: <20071031022806.138830@gmx.net>


> > Just searched a while for an option to bind epmd to a specific IP
> address. On my system it binds to all available IP addresses. Hmm, nothing so
> far. Do I really have to install a firewall because of epmd?
> > 
> > How do you deal with epmd? Any better options?

>  Mssrs. Google liked the following search request
> 
>   erlang min max epmd
>  
>   providing me http://thefactoryfactory.com/wordpress/?p=319 as the
>   third link.

Hi Michael,

thank's for your answer! Undocumented kernel parameters - very interesting! 
But it does not really solve the problem I have. inet_dist_listen_min and inet_dist_listen_max seem to limit the dynamic port range opened by erlang for incomming connections. Very usefull if you need to access Erlang through a firewall.

But what I need most is a way to make epmd listen on only a specified IP address on a server with more than one. Currently it listens on 0.0.0.0:4369 which means it listens on all configured interfaces. BTW: with the described parameters you cannot change the portmappers 4369 port.

Corporate security policies might very well forbid to open ports on interfaces which you do not need. Why should epmd listen on a LAN only used for drawing backups? And the typical security manager might also go nuts if I suggest to install firewalls on internal servers just because we cannot stop one process from listening on the wrong interfaces. I am currently dealing with such a situation.

But your link was interesting. I searched the kernel sources for these parameters and found another undocumented one: inet_dist_use_interface. After browsing through inet_tcp_dist.erl it turned out that it works in combination with the two mentioned parameters. So you can let Erlang only listen on a specific IP address/port range by using: erl -sname test -kernel inet_dist_listen_min 4001 inet_dist_listen_max 4001 inet_dist_use_interface "{135,13,164,12}"

Good to know. But still, epmd listens on 0.0.0.0:4369. Any other suggestions?

Regards,
Eric
-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail


From fritchie@REDACTED  Wed Oct 31 04:14:15 2007
From: fritchie@REDACTED (Scott Lystig Fritchie)
Date: Tue, 30 Oct 2007 22:14:15 -0500
Subject: [erlang-questions] beginner: Updating Data Structures
In-Reply-To: Message of "Mon, 29 Oct 2007 12:26:05 CDT."
	<005201c81a50$d56e9880$891ea8c0@SSI.CORP> 
Message-ID: <200710310314.l9V3EFcr075998@snookles.snookles.com>

>>>>> "dm" == David Mercer <dmercer@REDACTED> writes:

dm> To repeat my question, gentle readers, how ought I structure my
dm> data?  Is there an approach that I am missing that is both
dm> upgrade-friendly and ETS/Mnesia-compatible?

mnesia:transform_table/3 and /4 has worked well for us with only a
slight amount of staging tap-dance.

-Scott


From fritchie@REDACTED  Wed Oct 31 04:20:58 2007
From: fritchie@REDACTED (Scott Lystig Fritchie)
Date: Tue, 30 Oct 2007 22:20:58 -0500
Subject: [erlang-questions] beginner: mnesia secondary indices with
	ordered_set
In-Reply-To: Message of "Tue, 30 Oct 2007 11:54:21 BST."
	<8209f740710300354r3578ebfbt9a72a5c84dd8d955@mail.gmail.com> 
Message-ID: <200710310320.l9V3Kwv6076016@snookles.snookles.com>

>>>>> "uw" == Ulf Wiger <ulf@REDACTED> writes:

uw> No, Mnesia has only one type of index.

Yes, and it's implemented by an ETS bag, which won't give you the
efficient prefix search that Paul's looking/hoping for.

uw> The rdbms contrib in
uw> Jungerl implements index tables a bit differently (as full mnesia
uw> tables), and supports ordered index, among other things.
uw>
uw> But to the best of my knowledge, rdbms is not actively used by
uw> anyone (unless Rudolph van Graan is still using it).

I had some deadline pressure and couldn't rework our app to use rdbms
whole-heartedly.  :-(  But, back then, I adapted the technique that
rdbms uses for secondary indexing, and it works very well.  Many
thanks to Ulf and Rudolph for their guidance and suggestions.

-Scott


From saleyn@REDACTED  Wed Oct 31 06:48:54 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 31 Oct 2007 00:48:54 -0500
Subject: [erlang-questions] port_close/1 problem
Message-ID: <47281746.3@gmail.com>

Hi,

I am having a problem shutting down a port program started using 
open_port({spawn, Exe}, [binary, {packet, 2}, nouse_stdio, exit_status]).

When the owner Pid dies or explicitly calls port_close(Port) the linked 
OS process remains running.  It is a 3rd party binary interpreter.  Is 
it because it might be intercepting SIGINT or some other signals?

Is there a way to figure out it's OS pid and shut it down or is there a 
"proper" way of killing it from within the emulator?  (doing 
os:cmd("pkill ExeName") is not an option as there might be many 
instances of ExeName running.  I looked through port_info/1 options but 
don't see a way of determining port's OS pid.

Serge


From md@REDACTED  Wed Oct 31 08:02:16 2007
From: md@REDACTED (Maximillian Dornseif)
Date: Wed, 31 Oct 2007 00:02:16 -0700 (PDT)
Subject: [erlang-questions]  beginner: implement a cache/memoization
Message-ID: <13503890.post@talk.nabble.com>


I have a function which takes 1-5 sec to compute and will be called about 15
times a minute. Since many of the queries occure repeated I was thinking of
caching the last 1000 results (Each about 200 bytes).

The simpelest solution would be to create a ringbuffer. What would be the
appropriate datastructure? A Queue? This would mean that for every cace
check I would have to call lists:to_list/1.

An alternative would be a "Real" LRU cache. Is there an elegant
datastructure for that? Or do I have to go with an ets table an a cache
expiry process?

Regards

Maximillian
-- 
View this message in context: http://www.nabble.com/beginner%3A-implement-a-cache-memoization-tf4723208.html#a13503890
Sent from the Erlang Questions mailing list archive at Nabble.com.



From md@REDACTED  Wed Oct 31 08:08:32 2007
From: md@REDACTED (Maximillian Dornseif)
Date: Wed, 31 Oct 2007 00:08:32 -0700 (PDT)
Subject: [erlang-questions] Re strict epmd to one IP address?
In-Reply-To: <20071031022806.138830@gmx.net>
References: <20071030221448.67090@gmx.net>
	<20071031000750.GS21717@delora.autosys.us>
	<20071031022806.138830@gmx.net>
Message-ID: <13503892.post@talk.nabble.com>



nindeu wrote:
>> > Just searched a while for an option to bind epmd to a specific IP
>> address. On my system it binds to all available IP addresses.

> But what I need most is a way to make epmd listen on only a specified IP
> address on a server with more than one. Currently it listens on
> 0.0.0.0:4369 which means it listens on all configured interfaces.

If you use FreeBSD you could use jail(8) to restrict epmd to a single
Interface/IP. Something like

jail / epmdjail 10.11.12.13 /usr/local/bin/epmd ....

Regards

Maximillian
-- 
View this message in context: http://www.nabble.com/Restrict-epmd-to-one-IP-address--tf4721681.html#a13503892
Sent from the Erlang Questions mailing list archive at Nabble.com.



From eonnen@REDACTED  Wed Oct 31 07:24:29 2007
From: eonnen@REDACTED (Erik A. Onnen)
Date: Tue, 30 Oct 2007 23:24:29 -0700
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <20071031022806.138830@gmx.net>
References: <20071030221448.67090@gmx.net>	<20071031000750.GS21717@delora.autosys.us>
	<20071031022806.138830@gmx.net>
Message-ID: <47281F9D.1010608@gmail.com>

nindeu@REDACTED wrote:
>>> Just searched a while for an option to bind epmd to a specific IP
>> address. On my system it binds to all available IP addresses. Hmm, nothing so
>> far. Do I really have to install a firewall because of epmd?
>>> How do you deal with epmd? Any better options?
> 
> Good to know. But still, epmd listens on 0.0.0.0:4369. Any other suggestions?
> 
> Regards,
> Eric

Looking at source for R11B-5, epmd_int.h explicitly sets the address of 
the socket struct to INADDR_ANY (or IN6ADDR_ANY_INI for ipv6). So you're 
really at the mercy of the OS. Certain BSDs will choose the "default" 
interface, Linux will choose all interfaces, can't speak for Windows or Mac.

-erik



From als@REDACTED  Wed Oct 31 08:55:09 2007
From: als@REDACTED (Anthony Shipman)
Date: Wed, 31 Oct 2007 18:55:09 +1100
Subject: [erlang-questions] R12 snapshots
In-Reply-To: <de422a40710301446s246997das99beb7c01b77b394@mail.gmail.com>
References: <d3520a2f0710261531y707cccadq5f15f839074b4b7b@mail.gmail.com>
	<Pine.LNX.4.64.0710301800250.23828@hermes-1.csi.cam.ac.uk>
	<de422a40710301446s246997das99beb7c01b77b394@mail.gmail.com>
Message-ID: <200710311855.09499.als@iinet.net.au>

When I make an RPM from the erlang distribution my install section looks like 
the following. It would be nice if the system were improved so that this 
isn't necessary.

%install
rm -rf $RPM_BUILD_ROOT
make INSTALL_PREFIX=$RPM_BUILD_ROOT install

#   Correct the symlinks
pushd $RPM_BUILD_ROOT/usr/local/bin
for f in dialyzer epmd erl erlc escript run_erl to_erl typer
do
    if [ -x $f ]
    then
	/bin/rm $f
	ln -s /usr/local/lib/erlang/bin/$f $f
    fi
done

cd ../lib/erlang/bin
if [ -x epmd ]
then
    /bin/rm epmd
    ln -s ../erts-5.5.5/bin/epmd epmd
fi

#   Correct the paths in the scripts
for f in erl start
do
    if [ -x $f ]
    then
	sed "s,$RPM_BUILD_ROOT,," <$f >$$
	mv $$ $f
	chmod 755 $f
    fi
done

popd



-- 
Anthony Shipman                    Mamas don't let your babies 
als@REDACTED                   grow up to be outsourced.


From bengt.kleberg@REDACTED  Wed Oct 31 09:16:28 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 31 Oct 2007 09:16:28 +0100
Subject: [erlang-questions] Erlang "jar" file?
In-Reply-To: <13501250.post@talk.nabble.com>
References: <13501250.post@talk.nabble.com>
Message-ID: <472839DC.9030403@ericsson.com>

greetings,

i wrote an escript (besea) that is an executable archive handler 
(add/extract/run/etc). it does need a working escript on the targert 
(and that means at least CEAN is installed).

is that something similar to what you need?


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-31 02:17, Bernhard Damberger wrote:
> Does erlang support the idea of an archived collection of compiled code, like
> a jar file? I know that Joe Armstrong wrote a tool called ear that creates
> an archive for stand alone erlang (see http://www.sics.se/~joe/sae.html).
> But I got the impression that this is not supported by the standard erlang
> distribution (or is no longer being actively developed).
> 
> All the interpreter really needs to do is to be able to load the .beam files
> in from an archived file (say zip format). So, can it currently do so?
> 
> _bernhard


From bengt.kleberg@REDACTED  Wed Oct 31 09:21:39 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 31 Oct 2007 09:21:39 +0100
Subject: [erlang-questions] how: string stream?
In-Reply-To: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>
References: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>
Message-ID: <47283B13.4030404@ericsson.com>

greetings,

there are at least two string streams that i know of. Ulf Wiger has one 
and I have one (string_io). his should be better.


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-31 02:46, YC wrote:
> Hi all -
> 
> are there something equivalent to a string stream in Erlang?  The io 
> module utilizes a IoDevice(), but it seems an IoDevice is either a file 
> or socket, and unclear how to extend it.
> 
> I'm also wondering how to implement the equivalent of chaining streams 
> together to perform layers of processing - any examples are appreciated.
> 
> Thanks,
> yc
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions


From bengt.kleberg@REDACTED  Wed Oct 31 10:06:12 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 31 Oct 2007 10:06:12 +0100
Subject: [erlang-questions] beginner: implement a cache/memoization
In-Reply-To: <13503890.post@talk.nabble.com>
References: <13503890.post@talk.nabble.com>
Message-ID: <47284584.3030004@ericsson.com>

greetings,

i was going to recommend loop data in a list but a quick search gave me 
this example for the Bernoulli function (found on 
http://en.literateprograms.org/Bernoulli_numbers_(Erlang)):

bernoulli(N) ->
     Name = bernoulli,
     case ets:info(Name) of
     undefined -> ets:new(Name, [public, named_table]);
     _ -> true
     end,
     case ets:lookup(Name, N) of
     [] ->
         Val = bernoulli_i(N),
         ets:insert(Name, {N, Val}),
         Val;
     [{N, Val}] -> Val
     end.

changing to another function and limiting the ets table to 1000 items is 
left as an exercise for the reader :-).


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-31 08:02, Maximillian Dornseif wrote:
> I have a function which takes 1-5 sec to compute and will be called about 15
> times a minute. Since many of the queries occure repeated I was thinking of
> caching the last 1000 results (Each about 200 bytes).
> 
> The simpelest solution would be to create a ringbuffer. What would be the
> appropriate datastructure? A Queue? This would mean that for every cace
> check I would have to call lists:to_list/1.
> 
> An alternative would be a "Real" LRU cache. Is there an elegant
> datastructure for that? Or do I have to go with an ets table an a cache
> expiry process?
> 
> Regards
> 
> Maximillian


From ulf.wiger@REDACTED  Wed Oct 31 10:28:28 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Wed, 31 Oct 2007 10:28:28 +0100
Subject: [erlang-questions] beginner: mnesia secondary indices
	with	ordered_set
In-Reply-To: <8209f740710300354r3578ebfbt9a72a5c84dd8d955@mail.gmail.com>
References: <Pine.LNX.4.52L2.0710291418360.15501@herbie>
	<8209f740710300354r3578ebfbt9a72a5c84dd8d955@mail.gmail.com>
Message-ID: <47284ABC.7090904@ericsson.com>

Ulf Wiger wrote:
> No, Mnesia has only one type of index.
> The rdbms contrib in Jungerl implements index tables a bit
> differently (as full mnesia tables), and supports ordered index,
> among other things.
> 
> But to the best of my knowledge, rdbms is not actively used
> by anyone (unless Rudolph van Graan is still using it).

(Replying to myself - sorry)

There is actually another component that supports ordered
indices: Erlhive. It even supports aggregated indices, and
does so without any mnesia patches too (one thing that was
a bit troublesome with rdbms).

Of course, erlhive goes pretty far in abstracting away
mnesia as we know it. You're not allowed to use the
mnesia API directly, and the data model is deliberately
restrictive. On the flip side, it allows you to do things
inside mnesia transactions that would be insane to try
by hand (spawn, send, receive, ets tables, ...).

Indexing is done in a manner similar to that of rdbms.
Aggregated indices can be accomplished through use of
arrays of arrays, where an index is attached at each
appropriate level. Array, in erlhive, is an associative
array, with ordered set semantics.

Transaction overhead is higher than in rdbms, and scalability
is much likely significantly worse (I haven't tested it).
Erlhive wasn't really intended to be used just as a database
component, but I've toyed with the idea of separating the
parts so that it could be.

http://erlhive.sourceforge.net

This module:
http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/examples/blog/src/flexiblog.erl?revision=57&view=markup
Uses aggregated indices in a multi-user blogging
back-end implementation. (Find on 'index' to see
where the index parts are declared).

It uses this module as index callback:
http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/examples/blog/src/blog_index.erl?revision=57&view=markup

BR,
Ulf W


From ulf.wiger@REDACTED  Wed Oct 31 10:34:14 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Wed, 31 Oct 2007 10:34:14 +0100
Subject: [erlang-questions] how: string stream?
In-Reply-To: <47283B13.4030404@ericsson.com>
References: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>
	<47283B13.4030404@ericsson.com>
Message-ID: <47284C16.8010008@ericsson.com>

Bengt Kleberg wrote:
> greetings,
> 
> there are at least two string streams that i know of. Ulf Wiger has one 
> and I have one (string_io). his should be better.

I assume you mean this:

http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_ram_file_io_server.erl?revision=57&view=markup

used in this fasion:

http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_rewrite.erl?revision=64&view=markup

   121 parse_string(Str, Fname) ->
   122     {ok, Fd} = open_ram_file(Fname),
   123     file:write(Fd, Str),
   124     file:position(Fd, 0),
   125     case epp_dodger:parse(Fd) of
   126 	{ok, Tree} ->
   127 	    close_ram_file(Fd),
   128 	    Tree;
   129 	Error ->
   130 	    close_ram_file(Fd),
   131 	    erlang:error(Error)
   132     end.
   133
   134
   135 open_ram_file(Fname) ->
   136     erlhive_ram_file_io_server:start(self(), Fname, [read,write]).
   137
   138 close_ram_file(Fd) ->
   139     file:close(Fd).

It's not exactly a string stream, though.

BR,
Ulf W


From saleyn@REDACTED  Wed Oct 31 14:05:50 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 31 Oct 2007 08:05:50 -0500
Subject: [erlang-questions] port_close/1 problem
In-Reply-To: <47281746.3@gmail.com>
References: <47281746.3@gmail.com>
Message-ID: <47287DAE.4060901@gmail.com>

I reread this email and realized that I forgot to mention the fact that 
in my understanding port_close/1 merely closes its end of the pipe used 
to communicate with the port process.  So in this case if the running 
port program is in the middle of a blocking call it won't detect the 
closing of the file descriptor it uses to communicate with Erlang and 
will continue running.  So if I wanted to kill that port by sending an 
appropriate signal from Erlang how can I determine the OS pid of the port?

The obvious answer is to communicate that OS pid back to Erlang through 
the pipe, but in this particular case it takes over two to three minutes 
before the port loads all components and starts reading its end of the 
pipe, and I'd like to be able to kill it prior to that.

Any suggestions?

Serge

Serge Aleynikov wrote:
> Hi,
> 
> I am having a problem shutting down a port program started using 
> open_port({spawn, Exe}, [binary, {packet, 2}, nouse_stdio, exit_status]).
> 
> When the owner Pid dies or explicitly calls port_close(Port) the linked 
> OS process remains running.  It is a 3rd party binary interpreter.  Is 
> it because it might be intercepting SIGINT or some other signals?
> 
> Is there a way to figure out it's OS pid and shut it down or is there a 
> "proper" way of killing it from within the emulator?  (doing 
> os:cmd("pkill ExeName") is not an option as there might be many 
> instances of ExeName running.  I looked through port_info/1 options but 
> don't see a way of determining port's OS pid.
> 
> Serge
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
> 



From per@REDACTED  Wed Oct 31 13:13:39 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 31 Oct 2007 13:13:39 +0100 (CET)
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <20071031022806.138830@gmx.net>
Message-ID: <200710311213.l9VCDdj6017619@pluto.hedeland.org>

nindeu@REDACTED wrote:
>
>thank's for your answer! Undocumented kernel parameters - very interesting! 

>But your link was interesting. I searched the kernel sources for these
> parameters and found another undocumented one: inet_dist_use_interface.

Actually they're all documented in kernel(3) (where one might expect
them to be:-).

>Good to know. But still, epmd listens on 0.0.0.0:4369. Any other suggestions?

The documentation happens to be correct for epmd too:-) - there is no
way to tell it to bind to a specific address. Given the existence of
inet_dist_use_interface for the VM itself, it would make sense that epmd
had something equivalent (not necessarily calling an IP address an
"interface" though:-) - maybe contribute a patch?

--Per Hedeland


From francesco@REDACTED  Wed Oct 31 13:16:53 2007
From: francesco@REDACTED (Francesco Cesarini)
Date: Wed, 31 Oct 2007 12:16:53 +0000
Subject: [erlang-questions] Reminder: Obfuscated Erlang Competition deadline
	on Friday!
Message-ID: <47287235.4080107@erlang-consulting.com>

Hi All,

just to remind you that the deadline for the Obfuscated Erlang 
competition is approaching. To enter, read the instructions on

http://www.erlang-consulting.com/obfuscatederlang.html

Winners will be presented at the Erlang User Conference and solutions 
will be posted on this site. If you can not attend, we will send the 
prize by post.  Make sure you send your submissions by Friday to 
euc07@REDACTED <mailto:euc07@REDACTED>. The 
Jury will consist of Joe Armstrong, Richard Carlson (Last year's winner) 
and Jan Henry Nystrom.

Make sure that any submissions have been acknowledged. If not, resend 
them! If you have any questions, let us know.

See you @ EUC,
Francesco
--
http://www.erlang-consulting.com


From bengt.kleberg@REDACTED  Wed Oct 31 13:30:23 2007
From: bengt.kleberg@REDACTED (Bengt Kleberg)
Date: Wed, 31 Oct 2007 13:30:23 +0100
Subject: [erlang-questions] how: string stream?
In-Reply-To: <47284C16.8010008@ericsson.com>
References: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>
	<47283B13.4030404@ericsson.com> <47284C16.8010008@ericsson.com>
Message-ID: <4728755F.5000609@ericsson.com>

thank you for the link. i am not sure that is what i meant, though.

some time ago (1 year?) there was a repeat on this list of a 
string-as-io-object discussion that happened 3(?) years ago. i think you 
wrote that you had a ram-file module that could do this whereas i 
recommended my string_io. you convinced me that ram-file was a superset 
of string_io since it could do both files and strings.

i could be mistaken.


bengt
Those were the days...
    EPO guidelines 1978: "If the contribution to the known art resides
    solely in a computer program then the subject matter is not
    patentable in whatever manner it may be presented in the claims."


On 2007-10-31 10:34, Ulf Wiger (TN/EAB) wrote:
> Bengt Kleberg wrote:
>> greetings,
>>
>> there are at least two string streams that i know of. Ulf Wiger has 
>> one and I have one (string_io). his should be better.
> 
> I assume you mean this:
> 
> http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_ram_file_io_server.erl?revision=57&view=markup 
> 
> 
> used in this fasion:
> 
> http://erlhive.svn.sourceforge.net/viewvc/erlhive/trunk/lib/erlhive/src/erlhive_rewrite.erl?revision=64&view=markup 
> 
> 
>   121 parse_string(Str, Fname) ->
>   122     {ok, Fd} = open_ram_file(Fname),
>   123     file:write(Fd, Str),
>   124     file:position(Fd, 0),
>   125     case epp_dodger:parse(Fd) of
>   126     {ok, Tree} ->
>   127         close_ram_file(Fd),
>   128         Tree;
>   129     Error ->
>   130         close_ram_file(Fd),
>   131         erlang:error(Error)
>   132     end.
>   133
>   134
>   135 open_ram_file(Fname) ->
>   136     erlhive_ram_file_io_server:start(self(), Fname, [read,write]).
>   137
>   138 close_ram_file(Fd) ->
>   139     file:close(Fd).
> 
> It's not exactly a string stream, though.
> 
> BR,
> Ulf W


From per@REDACTED  Wed Oct 31 13:42:26 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 31 Oct 2007 13:42:26 +0100 (CET)
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <47281F9D.1010608@gmail.com>
Message-ID: <200710311242.l9VCgQTd018219@pluto.hedeland.org>

"Erik A. Onnen" <eonnen@REDACTED> wrote:
>
>nindeu@REDACTED wrote:
>>>> Just searched a while for an option to bind epmd to a specific IP
>>> address. On my system it binds to all available IP addresses. Hmm, nothing so
>>> far. Do I really have to install a firewall because of epmd?
>>>> How do you deal with epmd? Any better options?
>> 
>> Good to know. But still, epmd listens on 0.0.0.0:4369. Any other suggestions?
>> 
>> Regards,
>> Eric
>
>Looking at source for R11B-5, epmd_int.h explicitly sets the address of 
>the socket struct to INADDR_ANY (or IN6ADDR_ANY_INI for ipv6). So you're 
>really at the mercy of the OS. Certain BSDs will choose the "default" 
>interface, Linux will choose all interfaces, can't speak for Windows or Mac.

Hm, INADDR_ANY really does instruct the stack to accept a connection to
*any* of the locally configured addresses, (maybe it should have been
called INADDR_ALL:-) - if it doesn't, it's a bug, I've never seen that
on any BSD (or anywhere else). Perhaps you were actually using a
firewall, or something like a FreeBSD "jail" as mentioned in another
post (a jail by design never has more than one IP address).

And btw, at least on the Unices I have used, binding to an IP address
does not bind to an interface - if you bind to an address configured on
eth0 and a connection to it happens to arrive on eth1, it will happily
be accepted anyway (thus, depending on network topology and perceived
threats, binding to a specific address may be a pretty weak protection
and no substitute for a firewall - which is not to imply that epmd needs
protecting:-).

--Per Hedeland



From ulf.wiger@REDACTED  Wed Oct 31 13:46:04 2007
From: ulf.wiger@REDACTED (Ulf Wiger (TN/EAB))
Date: Wed, 31 Oct 2007 13:46:04 +0100
Subject: [erlang-questions] how: string stream?
In-Reply-To: <4728755F.5000609@ericsson.com>
References: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>	<47283B13.4030404@ericsson.com>
	<47284C16.8010008@ericsson.com> <4728755F.5000609@ericsson.com>
Message-ID: <4728790C.9050007@ericsson.com>

Bengt Kleberg wrote:
> thank you for the link. i am not sure that is what i meant, though.
> 
> some time ago (1 year?) there was a repeat on this list of a 
> string-as-io-object discussion that happened 3(?) years ago. i think you 
> wrote that you had a ram-file module that could do this whereas i 
> recommended my string_io. you convinced me that ram-file was a superset 
> of string_io since it could do both files and strings.
> 
> i could be mistaken.

That would be this post, I guess:

http://www.erlang.org/pipermail/erlang-questions/2007-March/025624.html

where I wrote, among other things:

(Ok, another drawback is that it's not a good solution
for parsing a byte stream, and one does need to copy
the string into the ram file.)

BR,
Ulf W  (:


From per@REDACTED  Wed Oct 31 13:53:25 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 31 Oct 2007 13:53:25 +0100 (CET)
Subject: [erlang-questions] port_close/1 problem
In-Reply-To: <47287DAE.4060901@gmail.com>
Message-ID: <200710311253.l9VCrPLc018467@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>I reread this email and realized that I forgot to mention the fact that 
>in my understanding port_close/1 merely closes its end of the pipe used 
>to communicate with the port process.

Correct - i.e. there are no signals sent.

>  So in this case if the running 
>port program is in the middle of a blocking call it won't detect the 
>closing of the file descriptor it uses to communicate with Erlang and 
>will continue running.  So if I wanted to kill that port by sending an 
>appropriate signal from Erlang how can I determine the OS pid of the port?
>
>The obvious answer is to communicate that OS pid back to Erlang through 
>the pipe, but in this particular case it takes over two to three minutes 
>before the port loads all components and starts reading its end of the 
>pipe, and I'd like to be able to kill it prior to that.

Hm, but after it has done that loading, this "3rd party binary
interpreter" will actually start communicating with Erlang on fd 3/4,
using {packet, 2} format? I guess not, i.e. you probably have some
wrapper around it - and if so, can't you send the pid from the wrapper
code *before* doing all the loading?

Alternatively, you could experiment with something along the lines of

 {spawn, Exe ++ " & echo $!"}

That obviously won't work as-is with {packet, 2}, so it has to be a bit
more complex, and the backgrounding *might* mess up the pipe plumbing,
but I don't really think it should.

--Per Hedeland


From jerith@REDACTED  Mon Oct 29 08:58:48 2007
From: jerith@REDACTED (Jeremy Thurgood)
Date: Mon, 29 Oct 2007 09:58:48 +0200
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <CB5291EC-C9FC-4669-81F4-4E9304FB0E49@ketralnis.com>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>	<992686.15039.qm@web38809.mail.mud.yahoo.com>	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>	<006a01c81986$1a359390$e751d854@Dell>	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<CB5291EC-C9FC-4669-81F4-4E9304FB0E49@ketralnis.com>
Message-ID: <472592B8.2080607@jerith.za.net>

David King wrote:
>> On 10/28/07, Matthew O'Gorman <mogorman@REDACTED> wrote:
>>> all the more reason you should have one stateside preferably in the
>>> south where i live ^_^
>> There are some really nice convention halls in Atlanta GA and  
>> Orlando FL...  =)
> 
> And the Bay Area, CA

While we're nominating venues, Cape Town is very nice this time of year. ;-)

--J


From saleyn@REDACTED  Wed Oct 31 15:49:56 2007
From: saleyn@REDACTED (Serge Aleynikov)
Date: Wed, 31 Oct 2007 09:49:56 -0500
Subject: [erlang-questions] port_close/1 problem
In-Reply-To: <200710311253.l9VCrPLc018467@pluto.hedeland.org>
References: <200710311253.l9VCrPLc018467@pluto.hedeland.org>
Message-ID: <47289614.9030107@gmail.com>

Per Hedeland wrote:
> Hm, but after it has done that loading, this "3rd party binary
> interpreter" will actually start communicating with Erlang on fd 3/4,
> using {packet, 2} format? 

Yes, if everything goes well in the initialization step.

> I guess not, i.e. you probably have some
> wrapper around it - and if so, can't you send the pid from the wrapper
> code *before* doing all the loading?

I don't have a wrapper - the interpreter does some heavy weight 
initialization (loading a bunch (~ 200!!!) of shared objects) and at 
some point will evaluate my script that would communicate with Erlang 
through a pipe.  If anything goes wrong I'd like to be able to kill the 
beast by sending a fatal signal (and it would not be desirable to 
introduce some light-weight middleman process that would sit between 
Erlang and the interpreter and do all intermediate pipe marshaling).

> Alternatively, you could experiment with something along the lines of
> 
>  {spawn, Exe ++ " & echo $!"}
> 
> That obviously won't work as-is with {packet, 2}, so it has to be a bit
> more complex, and the backgrounding *might* mess up the pipe plumbing,
> but I don't really think it should.

Thanks, I'll try, though I am doubtful that sending a process in the 
background won't hurt the piping setup.

Serge


From vances@REDACTED  Wed Oct 31 14:54:58 2007
From: vances@REDACTED (Vance Shipley)
Date: Wed, 31 Oct 2007 09:54:58 -0400
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <472592B8.2080607@jerith.za.net>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<CB5291EC-C9FC-4669-81F4-4E9304FB0E49@ketralnis.com>
	<472592B8.2080607@jerith.za.net>
Message-ID: <20071031135457.GE1848@little-black-book.motivity.ca>

If you'd like to hold it in St. Lucia, West Indies, I'll handle
setting up some facilities down here.

	-Vance

On Mon, Oct 29, 2007 at 09:58:48AM +0200, Jeremy Thurgood wrote:
}  While we're nominating venues, Cape Town is very nice this time of year. ;-)


From luna@REDACTED  Wed Oct 31 16:00:04 2007
From: luna@REDACTED (Daniel Luna)
Date: Wed, 31 Oct 2007 16:00:04 +0100 (CET)
Subject: [erlang-questions] EUC 2007 oversubscribed
In-Reply-To: <472592B8.2080607@jerith.za.net>
References: <4d08db370710280021ja85fcd0s5ac8bba4c4d2fd91@mail.gmail.com>
	<992686.15039.qm@web38809.mail.mud.yahoo.com>
	<d3520a2f0710280612i3583377fxd2699105cd034f7f@mail.gmail.com>
	<4d08db370710280732g3c784a9bnc0873a0a8c37b155@mail.gmail.com>
	<006a01c81986$1a359390$e751d854@Dell>
	<f49fd40d0710281024t2ca556b9w1395fd6e19f5288d@mail.gmail.com>
	<811f2f1c0710281141m4a69e6ddge4f94881b986e757@mail.gmail.com>
	<CB5291EC-C9FC-4669-81F4-4E9304FB0E49@ketralnis.com>
	<472592B8.2080607@jerith.za.net>
Message-ID: <Pine.LNX.4.64.0710311550150.20662@Psilocybe.Update.UU.SE>

On Mon, 29 Oct 2007, Jeremy Thurgood wrote:
> David King wrote:
>>> On 10/28/07, Matthew O'Gorman <mogorman@REDACTED> wrote:
>>>> all the more reason you should have one stateside preferably in the
>>>> south where i live ^_^
>>> There are some really nice convention halls in Atlanta GA and
>>> Orlando FL...  =)
>> And the Bay Area, CA
> While we're nominating venues, Cape Town is very nice this time of year. ;-)

And so is Uppsala, Sweden (about 65 kilometers from Stockholm...). But I 
will not propose a move of EUC.

It is my firm belief that the EUC will "always" be at Ericsson in ?lvsj?, 
but I hope that the ?lvsj? EUC will not be the only Erlang User 
Conference. There are Erlang users all over the world.

If one or more of you EUC-wannabes wish to start a tradition in some other 
part of the world I think we all would be very positive. And it would be a 
good excuse for me to travel...

Just my $0.02.

/Luna
-- 
Daniel Luna                           | Top reasons that I have a beard:
luna@REDACTED                     |  a) Laziness.
http://www.update.uu.se/~luna/        |  b) I can.
Don't look at my homepage (it stinks).|  c) I can get away with it.

From per@REDACTED  Wed Oct 31 17:38:23 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 31 Oct 2007 17:38:23 +0100 (CET)
Subject: [erlang-questions] port_close/1 problem
In-Reply-To: <47289614.9030107@gmail.com>
Message-ID: <200710311638.l9VGcNUl022987@pluto.hedeland.org>

Serge Aleynikov <saleyn@REDACTED> wrote:
>
>Per Hedeland wrote:
>> I guess not, i.e. you probably have some
>> wrapper around it - and if so, can't you send the pid from the wrapper
>> code *before* doing all the loading?
>
>I don't have a wrapper - the interpreter does some heavy weight 
>initialization (loading a bunch (~ 200!!!) of shared objects) and at 
>some point will evaluate my script that would communicate with Erlang 
>through a pipe. 

Ah, OK.

>> Alternatively, you could experiment with something along the lines of
>> 
>>  {spawn, Exe ++ " & echo $!"}
>> 
>> That obviously won't work as-is with {packet, 2}, so it has to be a bit
>> more complex, and the backgrounding *might* mess up the pipe plumbing,
>> but I don't really think it should.
>
>Thanks, I'll try, though I am doubtful that sending a process in the 
>background won't hurt the piping setup.

Actually I'm pretty sure it would work, the backgrounded process should
just inherit the pipe file descriptors - and a quick test seems to
confirm it. But you definitely lose the exit_status as your interpreter
process is no longer a child of the VM process. So, here's a wrapper for
you, with "more complex" and all:

1> Exe = "sleep 1000".
"sleep 1000"
2> Wrapper = "echo $$ | awk '{printf \"%c%c%s\",0,length($1),$1}' >&4; exec ".
"echo $$ | awk '{printf \"%c%c%s\",0,length($1),$1}' >&4; exec "
3> Settings = [binary, {packet, 2}, nouse_stdio, exit_status].
[binary,{packet,2},nouse_stdio,exit_status]
4> Port = open_port({spawn, Wrapper ++ Exe}, Settings).
#Port<0.95>
5> Pid = receive {Port, {data, P}} -> binary_to_list(P) end.
"97122"
6> os:cmd("kill " ++ Pid).
[]
7> Status = receive {Port, {exit_status, S}} -> S end.
143
8> 

--Per


From nindeu@REDACTED  Wed Oct 31 19:02:01 2007
From: nindeu@REDACTED (nindeu@REDACTED)
Date: Wed, 31 Oct 2007 19:02:01 +0100
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <200710311242.l9VCgQTd018219@pluto.hedeland.org>
References: <200710311242.l9VCgQTd018219@pluto.hedeland.org>
Message-ID: <20071031180201.184410@gmx.net>

Hi,

Thanks a lot for all your answers! Per, you are right, these parameters are documented. I just grepped through the doc and not through the lib directory.

It would be no problem to create a patch for epmd. But that creates dependencies for future Erlang updates. Not good.

Creating a jail for epmd is better than setting up a firewall. But nevertheless, jail will hardly be in the baseline image used to setup servers here. Security management will ask some interesting questions. But ok... Have to look how this is done best under Solaris and Linux.

Nevertheless in the future having an option for epmd to bind it to a specific IP address would be great. I guess epmd has challenged security policies already in other corporate environments.

Regards,
Eric
-- 
Psssst! Schon vom neuen GMX MultiMessenger geh?rt?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger


From chsu79@REDACTED  Wed Oct 31 20:15:31 2007
From: chsu79@REDACTED (Christian S)
Date: Wed, 31 Oct 2007 20:15:31 +0100
Subject: [erlang-questions] how: can I get live keyboard input in the
	Erlang shell?
In-Reply-To: <9b08084c0710261102v2775f288rb598afc706dc4edc@mail.gmail.com>
References: <9b08084c0710261102v2775f288rb598afc706dc4edc@mail.gmail.com>
Message-ID: <bc5c855a0710311215g4d6de5e4y90d6356e83d51ede@mail.gmail.com>

I remember from my C programming days that there is a unix terminal
control standard
called termios for modifying unix ttys regarding these things.

There appears to be some code in jungerl in the 'sl' library that does
this. I think one
would roughly do this:

{ok, TTY} = sl:open("/dev/tty", [{echo, false}, {mode, raw}])

And input seems to be delivered as messages. It has Tony Rogvall's
signature on it.

2007/10/26, Joe Armstrong <erlang@REDACTED>:
> There we go, I'm using my subject line tagging
>
> Once upon a long time ago I knew how to do this (I wrote the code) -
> but it was a long time ago
> and the code has probably been rewritten many times. Is there a
> defined way of doing this?
>
> It would be nice to turn echoing off (say for inputting passwords) and
> respond to *every* character.
> While we're at it it would be nice to use curses commands to move the
> cursor round the terminal.
>
> Is this possible or do I have to rewrite the terminal IO ?
>
> /Joe
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From yinso.chen@REDACTED  Wed Oct 31 20:15:19 2007
From: yinso.chen@REDACTED (YC)
Date: Wed, 31 Oct 2007 12:15:19 -0700
Subject: [erlang-questions] how: string stream?
In-Reply-To: <4728790C.9050007@ericsson.com>
References: <779bf2730710301846o1f9fd82dqe2f596f84ccb82d5@mail.gmail.com>
	<47283B13.4030404@ericsson.com> <47284C16.8010008@ericsson.com>
	<4728755F.5000609@ericsson.com> <4728790C.9050007@ericsson.com>
Message-ID: <779bf2730710311215g5e2fffa6q3c3c02e140d9055d@mail.gmail.com>

Thanks Bengt & Ulf for your responses and links.

Cheers,
yc

On 10/31/07, Ulf Wiger (TN/EAB) <ulf.wiger@REDACTED> wrote:
>
> Bengt Kleberg wrote:
> > thank you for the link. i am not sure that is what i meant, though.
> >
> > some time ago (1 year?) there was a repeat on this list of a
> > string-as-io-object discussion that happened 3(?) years ago. i think you
> > wrote that you had a ram-file module that could do this whereas i
> > recommended my string_io. you convinced me that ram-file was a superset
> > of string_io since it could do both files and strings.
> >
> > i could be mistaken.
>
> That would be this post, I guess:
>
> http://www.erlang.org/pipermail/erlang-questions/2007-March/025624.html
>
> where I wrote, among other things:
>
> (Ok, another drawback is that it's not a good solution
> for parsing a byte stream, and one does need to copy
> the string into the ram file.)
>
> BR,
> Ulf W  (:
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20071031/1906cf36/attachment.htm>

From per@REDACTED  Wed Oct 31 20:56:53 2007
From: per@REDACTED (Per Hedeland)
Date: Wed, 31 Oct 2007 20:56:53 +0100 (CET)
Subject: [erlang-questions] Restrict epmd to one IP address?
In-Reply-To: <20071031180201.184410@gmx.net>
Message-ID: <200710311956.l9VJurug027157@pluto.hedeland.org>

nindeu@REDACTED wrote:
>
>Thanks a lot for all your answers! Per, you are right, these parameters
>are documented. I just grepped through the doc and not through the lib
>directory.

Actually I referred to the man directory (or
http://www.erlang.org/doc/man/kernel_app.html for the man-challenged)...

>It would be no problem to create a patch for epmd. But that creates
>dependencies for future Erlang updates. Not good.

...and said "contribute", not "create" - if you send it to the OTP
folks, it may get incorporated into the future Erlang updates.

--Per


From nem@REDACTED  Wed Oct 31 21:18:04 2007
From: nem@REDACTED (Geoff Cant)
Date: Thu, 01 Nov 2007 09:18:04 +1300
Subject: [erlang-questions] Erlang "jar" file?
In-Reply-To: <13501250.post@talk.nabble.com> (Bernhard Damberger's message of
	"Tue, 30 Oct 2007 18:17:30 -0700 (PDT)")
References: <13501250.post@talk.nabble.com>
Message-ID: <m1lk9jhwcj.fsf@aegir.wgtn.cat-it.co.nz>

Bernhard Damberger <bernied@REDACTED> writes:

> Does erlang support the idea of an archived collection of compiled code, like
> a jar file? I know that Joe Armstrong wrote a tool called ear that creates
> an archive for stand alone erlang (see http://www.sics.se/~joe/sae.html).
> But I got the impression that this is not supported by the standard erlang
> distribution (or is no longer being actively developed).
>
> All the interpreter really needs to do is to be able to load the .beam files
> in from an archived file (say zip format). So, can it currently do so?
>
> _bernhard

I tried building a zip loader as an exercise and it turned out not to be
too difficult. (http://git.erlang.geek.nz/?p=ssae.git;a=summary)

It probably needs to be taken further so that you can do something like
'erl -s ssae init -s yourmodule init' and only distribute ssae.beam.

It would be good if the people who are interested in SAE to put together
a wiki page detailing what they expect out of SAE - then we could see
how difficult it would be to provide that. My guess is that modifying
the runtime system to load beam files out of a zip section in the
runtime binary would be tricky - but maybe that's not the requirement.

Cheers,
-- 
Geoff Cant



From kenneth.lundin@REDACTED  Wed Oct 31 22:23:43 2007
From: kenneth.lundin@REDACTED (Kenneth Lundin)
Date: Wed, 31 Oct 2007 22:23:43 +0100
Subject: [erlang-questions] Erlang "jar" file?
In-Reply-To: <m1lk9jhwcj.fsf@aegir.wgtn.cat-it.co.nz>
References: <13501250.post@talk.nabble.com>
	<m1lk9jhwcj.fsf@aegir.wgtn.cat-it.co.nz>
Message-ID: <de422a40710311423y21f175d4qd5862bbd0838e76a@mail.gmail.com>

Hi,

Loading code from zip archives or whatever other archive format has
already been implemented but not released.

We tried code loading from zip files with focus on compatibility and
improved speed for
code loading (it is easier if you skip the compatibility or make the
user to choose between ordinary code loading as of today and archive
loading)
There was no speed improvement so we did not release it.
But now we are considering adding support for code loading from
archive files again,
not for the sake of improved speed but for the reduced number of files you need
to distribute.

This will most probably show up in a R12B-x release next year.

/Kenneth (Erlang/OTP team at Ericsson)

On 10/31/07, Geoff Cant <nem@REDACTED> wrote:
> Bernhard Damberger <bernied@REDACTED> writes:
>
> > Does erlang support the idea of an archived collection of compiled code, like
> > a jar file? I know that Joe Armstrong wrote a tool called ear that creates
> > an archive for stand alone erlang (see http://www.sics.se/~joe/sae.html).
> > But I got the impression that this is not supported by the standard erlang
> > distribution (or is no longer being actively developed).
> >
> > All the interpreter really needs to do is to be able to load the .beam files
> > in from an archived file (say zip format). So, can it currently do so?
> >
> > _bernhard
>
> I tried building a zip loader as an exercise and it turned out not to be
> too difficult. (http://git.erlang.geek.nz/?p=ssae.git;a=summary)
>
> It probably needs to be taken further so that you can do something like
> 'erl -s ssae init -s yourmodule init' and only distribute ssae.beam.
>
> It would be good if the people who are interested in SAE to put together
> a wiki page detailing what they expect out of SAE - then we could see
> how difficult it would be to provide that. My guess is that modifying
> the runtime system to load beam files out of a zip section in the
> runtime binary would be tricky - but maybe that's not the requirement.
>
> Cheers,
> --
> Geoff Cant
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://www.erlang.org/mailman/listinfo/erlang-questions
>


From bjt@REDACTED  Wed Oct 31 23:11:49 2007
From: bjt@REDACTED (Benjamin Tolputt)
Date: Thu, 01 Nov 2007 09:11:49 +1100
Subject: [erlang-questions] Erlang "jar" file?
In-Reply-To: <de422a40710311423y21f175d4qd5862bbd0838e76a@mail.gmail.com>
References: <13501250.post@talk.nabble.com>	<m1lk9jhwcj.fsf@aegir.wgtn.cat-it.co.nz>
	<de422a40710311423y21f175d4qd5862bbd0838e76a@mail.gmail.com>
Message-ID: <4728FDA5.9020301@pmp.com.au>

See comments embedded below...

Kenneth Lundin wrote:
> <snip>
> We tried code loading from zip files with focus on compatibility and
> improved speed for code loading (it is easier if you skip the 
> compatibility or make the user to choose between ordinary code 
> loading as of today and archive loading)
> There was no speed improvement so we did not release it.
> But now we are considering adding support for code loading from
> archive files again, not for the sake of improved speed but for the 
> reduced number of files you need to distribute.
>   
This is the primary reason for "stand-alone Erlang" for me (though, as I 
have mentioned, there are other aspects for persons I work with). The 
idea of statically linking in the required libraries (like "beam.dll" on 
the Win32 platform), and providing two files for deployment (archive of 
beams & single executable) will make Erlang much easier to distribute & 
deploy.

I realise the dream of a two file deployment is a long way off (if it is 
even on the map!), but the idea is so good I can't let go of it!
> This will most probably show up in a R12B-x release next year.
>   
So after the initial R12B release has settled down then?

--B.J.Tolputt


From jao@REDACTED  Wed Oct 31 03:35:18 2007
From: jao@REDACTED (Jack Orenstein)
Date: Tue, 30 Oct 2007 22:35:18 -0400
Subject: [erlang-questions] gen_server:call vs. restart
Message-ID: <3E72C1D0-84FF-4395-AE59-C309188DE6B0@geophile.com>

Suppose I have the following modules:

- A gen_server.

- A supervisor that restarts the gen_server when necessary.

- A test that invokes functions of the gen_server with  
gen_server:call (using a global name, not a PID).

The test keeps calling gen_server functions. A separate process in  
the test module occasionally causes the gen_server to crash.  After  
the gen_server crash, and before the supervisor restarts it, a  
gen_server:call will fail, throwing a noproc exception. What is the  
right way to bulletproof the test program against this occurrence?

I could catch the noproc, wait a short while and try the call again.  
This doesn't seem very elegant, and it would be quite a pain to have  
to do this on each gen_server:call invocation.

I could add another supervisor, to restart the main process that was  
killed by the noproc exception, but this would lose any state that  
the process had. Not a problem for my test, but this could be a  
problem in practice.

Is there a standard approach to dealing with this problem?

Jack Orenstein