<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Cambria;
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.bullet2, li.bullet2, div.bullet2
{mso-style-name:bullet2;
margin-top:3.0pt;
margin-right:0in;
margin-bottom:0in;
margin-left:.45in;
margin-bottom:.0001pt;
text-indent:-.15in;
mso-list:l0 level1 lfo8;
font-size:11.0pt;
font-family:"Cambria","serif";}
span.EmailStyle171
{mso-style-name:EmailStyle171;
font-family:"Cambria","serif";
font-variant:normal !important;
color:windowtext;
display:none;
text-transform:none;
font-weight:normal;
font-style:normal;
text-decoration:none none;
vertical-align:baseline;}
p.bullet1, li.bullet1, div.bullet1
{mso-style-name:bullet1;
margin-top:3.0pt;
margin-right:0in;
margin-bottom:0in;
margin-left:.3in;
margin-bottom:.0001pt;
text-indent:-.15in;
mso-list:l0 level1 lfo8;
font-size:11.0pt;
font-family:"Cambria","serif";}
p.jNorm, li.jNorm, div.jNorm
{mso-style-name:jNorm;
margin-top:3.0pt;
margin-right:0in;
margin-bottom:0in;
margin-left:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Cambria","serif";}
p.code, li.code, div.code
{mso-style-name:code;
margin-top:3.0pt;
margin-right:0in;
margin-bottom:0in;
margin-left:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Courier New";}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Cambria","serif";
font-variant:normal !important;
color:windowtext;
text-transform:none;
font-weight:normal;
font-style:normal;
text-decoration:none none;
vertical-align:baseline;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Cambria","serif";}
.MsoPapDefault
{mso-style-type:export-only;
margin-bottom:3.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:886837541;
mso-list-type:hybrid;
mso-list-template-ids:263600464 740080172 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-style-link:bullet1;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=jNorm>thanx for the very insightful response!<o:p></o:p></p><p class=jNorm>I particularly like “make the message itself the process”, i.e. spawn a graph for each message/task.<o:p></o:p></p><p class=jNorm><o:p> </o:p></p><p class=jNorm><o:p> </o:p></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Jesper Louis Andersen [mailto:jesper.louis.andersen@gmail.com] <br><b>Sent:</b> Wednesday, April 29, 2020 2:44 AM<br><b>To:</b> jdmeta@gmail.com<br><b>Cc:</b> Erlang (E-mail)<br><b>Subject:</b> Re: architectural patterns for unifying state from multiple upstream processes<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><div><div><p class=MsoNormal style='margin-bottom:3.0pt'><span style='font-family:"Arial","sans-serif"'>On Wed, Apr 29, 2020 at 6:47 AM <<a href="mailto:jdmeta@gmail.com">jdmeta@gmail.com</a>> wrote:<o:p></o:p></span></p></div></div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>below is a trivial example of this scenario where a top-level service is decomposed into xService and yService which run concurrently.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>however, while yService performs some sub-task concurrently, it ultimately needs the output of xService to finish this sub-task (in this contrived case, to know which downstream service to send a message to where said message is some union of the work of xService and yService).<o:p></o:p></p></div></div></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'>To me, this is the key dependency. Since yService needs the output of xService, There is a dependency x -> y. Hence, it is not clear to me why it has to be two services in the first place. I'd probably just spawn a process per work-unit at the top level and then let the code run sequentially inside those processes, performing the job of x, then y, then z. Assuming there are a lot of work-units coming through the system, this is eventually going to saturate your processing cores. <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'>In general, there are two patterns I tend to see in concurrent systems. One is where you have relatively few processes, and they then forward messages along "channels" forming a processing network that's relatively static. It is common if creating processes tend to be expensive (like the need for an OS-thread or something such, or if a new thread has considerable memory overhead). The other, you make the message itself into a process and spawn a process per message. If creating a new process is cheap, this is more common. Erlang likes the latter approach[0]. You don't have a single graph in which multiple messages are flowing. You have a million graphs in each of which a single message is flowing. And that graph is usually represented by tail-calling functions. Of course, you have to handle access to limited resources in such a system. But this can often be done by factoring through a manager-process of some kind handing out tokens whenever a resource is available.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'>The other reason a process-per-message approach could be alluring is that it often limits message communication. That tends to make for a simpler program where fewer things can fail along the way.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-family:"Arial","sans-serif"'>[0] Any language with cheap process creation tend to be viable. Go could also be used for this approach, even though it has access to a channel primitive.<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></div></body></html>