Thought of the day: was RE: Gen_server and Gen_fsm questions

[Mark Scandariato]

The protocol drivers just map <things expressible> to <services provided>.

If a particular driver can't ask for a particular service, then there isn't a problem.

If the server can't fulfill a particular request, it just says so. The protocol driver 
will have to deal with it (until the server is enhanced). That is, don't build into the 
driver the fact that the server can't do X - but be prepared to handle a reply of 'Not Yet'.

Mark.

casper2000a@REDACTED wrote:
> Hi Joe,
> 
> Good idea. Infact I like being generalization. One common protocol and use pluggable drivers to talk to 
> out side world. But how can we do that?
> 
> With the little experience I have, if I take a very simple example FTP, that have a lot of features that 
> HTTP doesn't have. For example "resume broken download". All the protocols have their own 
> different features from each other. With this in mind, if we have to develop a driver, it will again be a 
> full server, for example FTP server not a FTP driver. The main problem here is, which part will handle 
> the protocol logic? Since the the protocol between Driver and the Server is common, we cannot 
> make the Server to have that logic. If the communication if basic request - response, we could easily 
> do that. But as I can see, there're more than that.
> 
> I may be looking at a wrong direction. But I am eager to follow up the progress in this thread.
> 
> Cheers,
> Eranga
> 
> 
> 
> 
> 
> 
> Quoting \"Joe Armstrong (AL/EAB)\" : > 
> 
> 
> Great - I like BIG pictures - wide-screen surround sound - the works 
> give it to me ...
> 
> On my way to work I was thinking about Remote Procedure Calls (RPC\'s)
> and !! and HTTP and SIP and all that kind of stuff.
> 
> Why are there so many *different* formats and ways of doing the *same*
> thing?
> 
> I think we should use ONE format for everything - bear with me and I\'ll
> try to explain: 
> 
> Firstly, what is an RPC?
> 
> In Erlang to do and RPC you send something a message and wait for a reply.
> 
> rpc(Pid, Q) ->
> Pid ! {self(), Q},
> receive
> {Pid, Reply} ->
> Reply
> end.
> 
> How do you do this in http?
> 
> The URL: \"http://www.erlang.org/index.html\" is a neat way of saying
> \"open port 80 on the host www.erlang.org then send a GET HTTP 1.0 ...
> request
> to port 80 and wait for a reply\"
> 
> ie \"http://www.erlang.org/index.html\" serves to name an RPC, it\'s very
> neat since it
> manages to say several things in one simple string.
> 
> Now let\'s imagine an Erlang equivalent: 
> 
> What might: \"http://www.erlang.org/Mod/Func?Arg1=val1&Arg2=Val2\" mean?
> 
> Let\'s interpret this as:
> 
> Go to port 80 on www.erlang.org and write a GET Mod/Fun?Arg1=Val1&
> ...
> string to the port.
> 
> What does the server do? - Yes - evaluate Mod:Func(Args) ... assume
> this
> returns a term T - then convert this to a binary and send it back with a
> mime type
> text/erlangBinterm.
> 
> Now suppose I want to make \"something like email\" based on http -
> easy! define a URL like
> 
> \"http://www.erlang.org/mail/deliver&who=joe&subject=mail....\"
> 
> To mean \"deliver mail to joe ....\"
> 
> Now structure your software like this:
> 
> HTTP GET ... +--------+ {[mail,deliver],[{\"ho\",\"joe\"}]}
> +--------+
> ---->------------| driver |-------->--------------------------| server
> |
> +--------+
> +--------+
> 
> The driver does HTTP packet reassembly etc - it parses the request
> into a normalised Erlang term and sends it to the server.
> 
> Why go to all this trouble?
> 
> Suppose we change transport medium - to FTP
> 
> \"ftp://joe@www.erlang.org/mail/deliver&who=joe&subject=mail....\"
> 
> Again this has to be interpreted and parsed, so we add a new
> front-end
> 
> 
> HTTP GET ... +--------+ {[mail,deliver],[{\"ho\",\"joe\"}]}
> +--------+
> ---->------------| driver |-------->---+-----------------------|
> server |
> +--------+ |
> +--------+
> |
> FTP put request +--------+ |
> ----->-----------| driver |----->------+
> +--------+
> 
> Now the back-end server only understands Erlang message - the drivers
> turn these 
> messages into HTTP, or FTP or whatever is the flavour of the day (even XML
> RPC)
> 
> 
> What we have to recognise is that all these different syntaxes are
> just different
> ways of doing an RPC.
> 
> 
> RPC format 1 +---------+ Universal term format
> +--------+
> ---->------------| driver1 |-------->---+-----------------------|
> server |
> +---------+ |
> +--------+
> |
> RPC format 2 +---------+ |
> ----->-----------| driver2 |----->------+
> +---------+ |
> |
> RPC format 3 +---------+ |
> ----->-----------| driver3 |----->------+
> +---------+
> 
> Now why people get all excited about the different formats (XML-RCP,
> SOAP,
> HTTP, FTP, sun-rpc) etc. is beyond me - THEY ARE ALL JUST DIFFERENT
> SYNTAXES
> FOR RPCs.
> 
> Whether you fetch a file with HTTP or FTP or rcp or XML-RPC is
> *irrelevant*
> the semantics \"fetching a file\" is identical.
> 
> Still with me? - good.
> 
> Let\'s generalise
> 
> P://H/Function?Args
> 
> means 
> 1) let\'s use a Protocol called P 
> 2) To talk to a host H
> 3) and tell it to do Function with arguments
> 4) Args
> 
> That\'s why it\'s a very nice notation (4 things in one string)
> 
> How do we find H? - there are three alternative.
> 
> 1) If *is* the server hostname then use DNS
> 2) If H contains no hostname use a distributed hash table (chord,
> pastry, DKS, CAN etc)
> 3) If H is a mixture of a server name and a key use SIP
> 
> To do 1) you need to own some DNMS domain that you can easily modify
> 2) is research - they are no public name severs (or am I wrong?). 3)
> Implies a
> SIP proxy at a fixed hostname. Given name@REDACTED the host bit can be
> resolved by DNS
> and the name bit can be resolved by a SIP server at host.
> 
> So here\'s an idea:
> 
> Lets define a new URL (or is it a URI - I can never remember)
> 
> erl://name@host/Mod/Func?arg1=val1&arg2=val2
> 
> To mean something like: use SIP to locate a joe@REDACTED, open a socket
> to an Erlang
> server on this machine and send it the message {rpc, From, \"Mod/Func\",
> [{\"arg1\",\"val1\"}..}]}
> 
> SIP stands for \"session initiation protocol\" - I assume the designers
> of
> SIP were thinking of \"Erlang sessions\". I guess SIP is really just \"a
> rather complicated
> way of connecting two Erlang processes together\" - once you\'ve done this
> then the
> processes can get on with the real job of \"doing something useful.\"
> 
> This is, of course, the tricky bit - discussing syntax (should we use
> XML, HTTP, FTP, SIP, DNS, SOAP) distracts attention away from semantics
> (what should we do with this stuff).
> 
> The former question usually attracts much more attention than the
> latter :-)
> 
> Cheers
> 
> /Joe
> 
> that set me off,
> so I\'ll try a quick summary.
> 
> Get rid of proprietary formats/protocols etc. as soon as possible - use
> drivers to
> convert to a universal messaging format (Erlang terms). Write all your
> programs using
> the internal formats. Introduce a universal naming scheme for everything>
> 
> -----Original Message-----
> From: Casper [mailto:casper2000a@REDACTED]
> Sent: den 4 januari 2005 13:25
> To: Joe Armstrong (AL/EAB); \'Vance Shipley\'
> Cc: erlang-questions@REDACTED
> Subject: RE: Gen_server and Gen_fsm questions
> 
> 
> Hi Joe,
> 
> Wish you a happy new year too. And thanks a lot for your valuable advice.
> 
> I have a BIG picture. A picture of a common platform, very generalized,
> which has Telecom applications such as SMSC, IVR, Prepaid, HLR, SCP, etc
> as
> pluggable modules (or applications). One module to handle ISUP Call
> control,
> one module for IVR functions, one for Prepaid functions, one for TCAP, one
> for SMS handling, one OAM, etc., distributed and having full redundancy.
> 
> I\'m kind of tired by doing various platforms in various languages and
> platforms. MMSC runs on any, since it\'s done using Java, SMSC on Linux
> C/C++, IVR on Win32 VC, etc. These developments are started in different
> levels/times, so has not come under one platform. Also maintenance and
> debugging takes a lot of time. DBMS is not giving the required transaction
> speed, etc. So I want all of them to come under one platform, and I\'m
> getting very much convinced, under Erlang/OTP platform.
> 
> I know it\'ll be difficult to start, but I\'m sure it\'s worth doing. So at
> the
> moment I\'m investigating the architectures of other platforms developed
> using Erlang. It\'s kind of hard to find any good documentation of such a
> system. 
> 
> If any of you can give me any advice/reference materials regarding above
> discussion, I greatly appreciate.
> 
> Thanks!
> Eranga
> 
> 
> 
> 
> 
> 
> 
> -----Original Message-----
> From: Joe Armstrong (AL/EAB) [mailto:joe.armstrong@REDACTED] 
> Sent: Tuesday, January 04, 2005 4:16 PM
> To: \'Casper\'; \'Vance Shipley\'
> Cc: erlang-questions@REDACTED
> Subject: RE: Gen_server and Gen_fsm questions
> 
> 
>>If I have one/two process for each call, then if I maintain let\'s say
>>100,000 simultaneous calls, I will have to create 200,000 gen_fsm, ie.
>>Processes. 
> 
> 
> Yes
> 
> 
>>Is that a Good method?
> 
> 
> Yes
> 
> 
>>Will that create unnecessary system overhead?
> 
> 
> No
> 
> It\'s exactly the right way to think.
> 
> You have to get used to thinking in terms of processes - creating
> processes
> is a light-weight operation (this means you can create lot\'s of them very
> quickly).
> 
> No you might run into memory problems - I don\'t know what the minimum size
> of a process is
> but let\'s guess 1KB - so your 200 K process might take 200M of memory and
> that might
> be a problem.
> 
> But suppose you were to do it some other way - suppose you \"suspend\" a
> process when it\'s not
> doing anything useful - you have to store it\'s data structures somewhere -
> you have to
> make it go away, store it\'s data structures, then at a later stage wake it
> up and
> restore it\'s data structures etc. All of this takes lots of unnecessary
> code
> and there\'s no guarantee that it\'s quicker.
> 
> Even storing the data structures required by suspended processes takes
> space
> so doing this might not be a good idea.
> 
> The Erlang \"way\" is to identify all the truly parallel activities in your
> application and then assign exactly ONE process per activity. (The exactly
> ONE bit is important) -
> this makes the code isomorphic to the problem - and easy to write
> understand
> and debug.
> 
> So first you do as I have suggested - THEN you measure and possibly
> optimise.
> 
> First make it right - then make it fast.
> 
> Happy new Year
> 
> --------------This mail sent through OmniBIS.com--------------