[erlang-questions] Is Erlang a good tool for this particular project?

Wed Jul 13 20:03:08 CEST 2011

On Wed, Jul 13, 2011 at 7:16 AM, Matthew Hillsborough <
matthew.hillsborough@REDACTED> wrote:

> Hi Joe,
>
> Thanks for posting links to this. Surprised Joe A. did not mention it in
> his post, it has him listed as an author. :)
>
> Just from a quick glimpse, this appears to be using MochiWeb, which is
> completely HTTP based. I am not entirely sure if that's the route I am
> looking to go, unless I am overlooking something here. With that said, there
> are probably parts of this code I can pick out and re-use in a pure TCP
> socket implementation (if this is indeed purely HTTP, which I suspect it
> might be).
>
> Would love to hear I am wrong though so I do not need to reinvent the
> wheel.
>
> Thank you and happy hacking -
>

Out of curiosity, is there a reason you wouldn't use thrift or protocol
buffers? Even if you don't use their RPC mechanisms, you could use one of
those as the serialized form for message passing (over plain sockets, if you
so desire). I think a binary serialization format will still be more compact
than JSON and faster to deserialize as well.

Cheers,
Joel

>
> Matthew
>
> On Wed, Jul 13, 2011 at 9:43 AM, Joseph Norton <norton@REDACTED>wrote:
>
>>
>> Matt -
>>
>> There is already a client and server implementation similar to your idea
>> that uses JSON as the wire format over TCP/IP.  See the section "JSF" for a
>> brief description:
>>
>> http://norton.github.com/ubf/ubf-user-guide.en.html#_tcp_ip
>>
>> Download and building instructions are here:
>>
>> https://github.com/norton/ubf-jsonrpc
>>
>> Hope you find it helpful.
>>
>> - Joe N.
>>
>>
>> On Jul 13, 2011, at 9:42 PM, Matthew Hillsborough wrote:
>>
>> Hi Ale and everyone else who replied,
>>
>> That's exactly the point, I don't think HTTP is necessary at all! There is
>> the overhead of sending extraneous HTTP headers over the wire. All of those
>> additional bits take additional CPU time and bandwidth (on a mobile device
>> with limited CPU and even more limited bandwidth!). I see absolutely no need
>> for sending HTTP headers and parsing them from the response for this,
>> particularly because I am not building a web browser based application. I
>> have access to C/C++/Objective-C (on iOS) and Java (on Android) and these
>> are perfectly capable of working with TCP sockets. A friend of mine
>> suggested that I just pass messages to the server as JSON using a prefixed
>> header that specifies the length of the message. That would be it! Simple
>> and compact.
>>
>> I just wanted to validate the idea and see if others thing I'll run into
>> too many edge cases uses TCP sockets via an RPC type server to build this
>> server/client architecture out. I want to really make it reusable so all of
>> my mobile products can use it.
>>
>> Loving all the advice coming in! Thanks all.
>>
>> Matthew.
>>
>> On Tue, Jul 12, 2011 at 5:03 PM, Ale <peralta.alejandro@REDACTED> wrote:
>>
>>> Hi,
>>>
>>> Just a thought here... Erlang is great for scalability and handling
>>> multiple connections, and well I'm sure a lot of people can give you a
>>> better description fo this, but it occurred to me if you considered
>>> other protocols instead of HTTP? The problem seems to be there rather
>>> than the language you write your sever. It occurred to me that you
>>> might benefit from using Jabber/XMPP. Googling I found some iOS jabber
>>> clients http://code.google.com/p/xmppframework/ and the cannonical
>>> jabber server is written in Erlang.
>>>
>>> Regards,
>>>
>>> 2011/7/12 Matthew Hillsborough <matthew.hillsborough@REDACTED>:
>>> > Greetings Erlang community,
>>> > Let me further elaborate on my question that's in the subject of this
>>> > message. I tried to reach out with this question on StackOverflow,
>>> however I
>>> > did not have much luck there. Perhaps the community here can provide
>>> some
>>> > feedback here for me to let me know if I'm on the right track or if
>>> Erlang
>>> > is not the right tool for what I'm trying to accomplish.
>>> >
>>> > I'm building native mobile applications in both iOS and Android. These
>>> apps
>>> > require "realtime" updates from and to the server, same as many (but
>>> not
>>> > all) network-based application does (Facebook, social games like Words
>>> with
>>> > Friends, Finance applications, etc). The communication here is
>>> > bi-directional, in the sense that the server might have updates for the
>>> > mobile clients and the clients will be pushing data down to the server
>>> > whenever necessary.
>>> >
>>> > I think using HTTP long polling for this is over kill in the sense that
>>> long
>>> > polling can be detrimental to battery life, especially with a lot of
>>> TCP
>>> > setup/teardown for every HTTP connection the device needs to send out
>>> > through the wire. It might make sense to have the mobile applications
>>> use
>>> > persistent TCP sockets to establish a connection to the server, and
>>> send RPC
>>> > style commands to the server for all web service communication. This
>>> > ofcourse, would require a server to handle the long-lived TCP
>>> connection and
>>> > be able to speak to a web service once it makes sense of the data
>>> passed
>>> > down the TCP pipe. I'm thinking of passing data in plain text using
>>> JSON or
>>> > XML and then using some kind of Erlang interface to HTTP to call a web
>>> > service to handle all the REST type communication. The responses would
>>> then
>>> > go back to the "RPC" Erlang instance, which would send the updates to
>>> the
>>> > appropriate client(s).
>>> >
>>> > Perhaps an Erlang based RPC server would do well for a network based
>>> > application like this. It would allow for the mobile apps to send and
>>> > receive data from the server all over one connection without multiple
>>> > setup/tear down that individual HTTP requests would do. Since no web
>>> browser
>>> > is involved, we do not need to deal with the nuances of HTTP
>>> long-polling at
>>> > the mobile client level. I also haven't seen great long
>>> polling/keep-alive
>>> > support on the client-side in iOS, but that's irrelevant for the
>>> community
>>> > here.
>>> >
>>> > A lot of these "COMET" and long-polling/streaming servers are built
>>> with
>>> > HTTP in mind. I'm thinking just using a plain-text protocol over TCP is
>>> > better catered for the type of app I'm building, will make the client
>>> more
>>> > responsive, allow for receiving of updates from the server without
>>> > constantly polling the server, etc.
>>> >
>>> > I also looked into HTTP pipelining, but it doesn't look to be worth the
>>> > trouble when it comes to implementing it on the clients. Also, I'm not
>>> sure
>>> > if it would allow for bi-directional communication in the
>>> client<->server
>>> > communication channel.
>>> >
>>> > Am I completely out of line in thinking that building a custom solution
>>> in
>>> > Erlang is a good idea here? To my understanding, Erlang excels at
>>> servers
>>> > like this, and if I run the server on tcp/80, I should be able to avoid
>>> most
>>> > firewall/port issues. The client would need work to deal with timeouts,
>>> re
>>> > connections, acknowledging receipt of asynchronous requests, but that's
>>> not
>>> > Erlang's problem.
>>> >
>>> > Has anyone built something similar before? Should I just stick to a web
>>> > server and deal with "COMET" type technologies? (WebSockets,
>>> long-polling,
>>> > client-side polling).
>>> >
>>> > Was hoping someone could solidify that I'm not entirely insane for
>>> wanting a
>>> > better solution than HTTP would serve in this case, at least at the
>>> client
>>> > level. I'll still be using HTTP/REST extensively, the Erlang server
>>> would
>>> > just handle the persistent connections and messaging to the Web Service
>>> > (which would probably be something like Django or Rails).
>>> >
>>> > Sorry for the long post; I am just excited to get into the heads of
>>> people
>>> > who are smarter than I.
>>> >
>>> > Happy hacking!
>>> >
>>> > Matthew
>>> >
>>> > _______________________________________________
>>> > erlang-questions mailing list
>>> > erlang-questions@REDACTED
>>> > http://erlang.org/mailman/listinfo/erlang-questions
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Ale.
>>>
>>
>> _______________________________________________
>> erlang-questions mailing list
>> erlang-questions@REDACTED
>> http://erlang.org/mailman/listinfo/erlang-questions
>>
>>
>> Joseph Norton
>> norton@REDACTED
>>
>>
>>
>>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20110713/aa1d57af/attachment.htm>