[erlang-questions] shared records in .hrl files

[ANTHONY MOLINARO]

You could hide the record implementation inside a module, then only the module would have to know about differences between versions.  All the sub-systems use the functional interface of the module and the record is just passed in as the first argument (this is similar to how one would use struct’s in C).

So for instance

-module (widget).
-record (widget_v1, { field1 }).
-record (widget_v2, { field1, field2 }).
-export ([new/0, get/2, set/3]).

new () ->
  #widget_v2 {}.

get (#widget_v1 { field1 = Field1 }, field1) ->
  Field1;
get (#widget_v2 { field1 = Field1 }, field1) ->
  Field1;
get (#widget_v1 {}, field2) ->
  undefined;
get (#widget_v2 { field2 = Field2 }, field2) ->
  Field2..

set (Widget = #widget_v1 {}, field1, Field1) ->
  Widget#widget_v1 { field1 = Field1 };
set (Widget = #widget_v2 {}, field1, Field1) ->
  Widget#widget_v2 { field1 = Field1 };
set (Widget = #widget_v1 {}, field2, Field2) ->
  set (convert (Widget), field2, Field2);  % automatically convert v1 to v2 here
set (Widget = #widget_v2 {}, field2, Field2) ->
  Widget#widget_v2 { field2 = Field2 }.

convert (#widget_v1 { field1 = Field1 }) ->
  #widget_v2 { field1 = Field1 }.

The different components could just do things like

Widget = widget:set (widget:new(), field1, “foo”),
widget:get (Widget, field1),
…

Then if you need to add fields the actual implementation is hidden.

If you store a previous version of a record you can still pass it into the get/set functions and get back values or defaults.  All upgrade logic could be hidden as in the above example where a record is converted when you try to set a new field in an old version.   Now over time you could end up with many versions of your record, but all the logic for the different versions is hidden inside the module interface.

It’s maybe a bit OO, and I’d be curious if someone has a better idea that is more functional in nature.

-Anthony


On Jul 3, 2014, at 10:18 AM, Felix Gallo <felixgallo@REDACTED> wrote:

> Consider a system that receives application messages in an extensibly large number of formats (e.g. json, xml, yaml, pictograms, esperanto).  This system wishes to translate the messages into a unified data format, do various centralized bookkeeping tasks, and then pass those normalized messages off to workers for further processing.  The system further distrusts the application messages (and the format conversion libraries) and so wishes to spawn a worker for each message and let it crash if nonsense or tomfoolery is involved.
> 
> A picture:
> 
> Queue of Incoming Messages -> [Pool of Specialized Translators] -> [Processing Manager] -> [Pool of Workers]
> 
> The "natural" approach from a C / OO background is to define a layout for the unified format, perhaps as an object or struct, and share that across the specialized translators, the processing manager, and the worker processes.  And indeed that solves the problem fairly neatly here as well, except you can't walk the streets of Los Angeles without tripping over a hobo brought to desperate circumstances who regales you with his woeful cautionary tale of attempting to perform a live upgrade with a record definition shared across multiple modules in an hrl file.
> 
> After pondering it, it seems like you can choose at most two of { live upgrades, separation into multiple processes, simple and sensible records }.
> 
> Needing to choose live upgrades and separation into multiple processes, one then starts thinking of bloodcurdling ways to handle the shared record issue, including:
> 
> 1.  Treat record formats as immutable and just keep adding them ('mystate1', 'mystate2', ... 'mystate11'...) along with all of the accompanied handling.
> 2.  Keep a version field in every record format and start them out being as big as you might ever want them to get.
> 3.  Clone the OTP source repository and hack in record upgrade messages.
> 
> I'd be interested in hearing if anyone has any better ideas, or how others have handled this scenario in real world conditions.
> 
> F.
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions