[erlang-questions] shared records in .hrl files

[Dmitry Kolesnikov]

Hello,

I am afraid that massive amount of record definition will complicate the system.
Have you considered the use of map or key/val lists as unified format.

- Dmitry


On 03 Jul 2014, at 20:18, Felix Gallo <felixgallo@REDACTED> wrote:

> Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto).  This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing.  The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved.
> 
> A picture:
> 
> Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers]
> 
> The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes.  And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file.
> 
> After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }.
> 
> Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including:
> 
> 1.  Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling.
> 2.  Keep a version field in every record format and start them out being as big as you might ever want them to get.
> 3.  Clone the OTP source repository and hack in record upgrade messages.
> 
> I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions.
> 
> F.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions