[Original] Re: Solving the right problem

Marthin Laubscher marthin@REDACTED
Thu Nov 7 09:04:51 CET 2019

Thanks. I’m sorry for not making it clear that in my case the communication question is integral to my business logic. I certainly don’t want to abstract it away behind a general purpose tool which involves sequential fundamentals such as single threaded message queues. XMPP would inevitably come into play higher up in the application stack but I’d be hard-pressed to latch my design paradigm to the general messaging mindset. From my project’s perspective messaging would be of its many derivatives.


I’m interested in learning as much as possible from those who designed other more general purpose implementations such a you mentioned. I particularly found the RabbitMQ best practices article very useful. The general gist of those boils down to using your best understanding of your clients, data and resources to design your messages and message queues so bottlenecks are avoided as a result. That’s fine, when you know those things in advance which I simply don’t. Everything about my clients, data and resources would be constantly changing. I’d be a fool to try and predict any of it at design time and an idiot to then try impose my predictions on reality. But I am confident that I can abstract the nature of changes my clients, data and resources would changes would be subjected to well enough to design and build the code to measure and respond to those changes as and when they happen. In terms of the RabbitMQ best practices I mentioned, that would translate into adapting the design of my queues, nodes and messages back and forth based on transient, slow-changing and fast-acting changes – not what I’d call simple nor practical. 


I’ve managed to simplify the solution down to a single core datatype in which all data is stored, exchanged and referenced, ranging from a few bytes to as big as needed but with a large composite key. The obvious storage problem this introduces is overcome by applying the optimization techniques which Edgar Codd taught and many more dynamically at run-time based on measured meta-data as opposed to design time based on forecast-based modelling. While that may seem impossibly complex to pull off, it actually becomes brilliantly simple and easy once you’ve set up the environment at the appropriate level of abstraction. 



From: T Ty <tty.erlang@REDACTED>
Date: Wednesday, 6 November, 2019 at 11:04
To: "marthin@REDACTED" <marthin@REDACTED>
Cc: Erlang Questions <erlang-questions@REDACTED>
Subject: Re: [Original] Re: Solving the right problem


I would suggest you ignore the Erlang/Elixir mode of thought as a starting point.


Your architecture is closely related to either XMPP or AMQP protocol and suggest you base it off these instead of heading straight down do a coding language. Of course I would be bias towards MongooseIM or RabbitMQ. This abstracts the entire communication question away and you can focus on the business logic. 


In XMPP, each client device and server would be XMPP clients and both would communicate using simple text based coded messages. You automatically get federation and presence notification. Store and forward is automatically built in. XMPP is also moving towards addressing IoT devices which might further simplify your architecture.


In RabbitMQ each client/server would have a two queue (in/out) and all communications goes through these. Multiple clients can share a single queue pair if they are grouped together e.g. all clients within the same geo-location. 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20191107/df1b4068/attachment.htm>

More information about the erlang-questions mailing list