<html><head><style>body{font-family:Helvetica,Arial;font-size:12px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">“How to handle older largely inactive processes”</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"> What we do (did?) was to basically flush/hibernate the process. To generalize, if the process hasn’t “done” anything in a while, save its state somewhere, and then hibernate (which has the added benefit of dealing with GC issues :-) </div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Mind you, there is a wealth of info buried in the phrase “save its state somewhere”. This really, <i>really</i> depends on how big, how scalable, how fault-tolerant, how geographic, how….. you intend on getting. In short, it can range from “write out a text file” to “pay Riak gobs-o-money and have nodes worldwide”. YMMV</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">“Heroku gang” <— My apologies, I meant “those fine fine folks at Heroku who happen to be doing erlang”</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">cheers</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:12px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div> <div id="bloop_sign_1392674295941381888" class="bloop_sign"><div style="font-family:helvetica,arial;font-size:13px"><div style="color: rgb(34, 34, 34); line-height: normal; font-family: Helvetica; word-wrap: break-word;"><div style="margin: 0in 0in 0.0001pt;"><font color="#1f497d" face="Calibri, sans-serif"><span style="font-size: 15px;"><b><div style="font-style: italic; margin: 0px; font-family: Calibri;"><b style="color: rgb(17, 85, 204);"><a href="http://www.gravatar.com/avatar/204a87f81a0d9764c1f3364f53e8facf.png" target="_blank" style="color: rgb(17, 85, 204);">Mahesh Paolini-Subramanya</a></b></div><div style="margin: 0px; font-family: Calibri;"><span style="font-weight: normal;">That tall bald Indian guy..</span> <br>
</div></b></span></font></div></div><div style="color: rgb(34, 34, 34); line-height: normal; font-family: Helvetica; word-wrap: break-word;"><div style="margin: 0in 0in 0.0001pt;"><font color="#1f497d" face="Calibri, sans-serif"><span style="font-size: 15px;"><b><div style="margin: 0px; font-family: Calibri;"><div style="font-family: Helvetica; word-wrap: break-word;"><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;"><span style="font-size: 11pt; font-family: Calibri, sans-serif;"><div style="margin: 0px; font-family: Calibri; color: rgb(1, 108, 226);"><a href="https://plus.google.com/u/0/108074935470209044442/posts" target="_blank" style="color: rgb(17, 85, 204);">Google+</a><span style="color: rgb(31, 73, 125);"> | <a href="http://dieswaytoofast.blogspot.com/" target="_blank" style="color: rgb(17, 85, 204);"><span style="color: rgb(1, 108, 226);">Blog</span></a></span> <span style="color: rgb(31, 73, 125);"> | <span style="color: rgb(1, 108, 226);"><a href="https://twitter.com/dieswaytoofast" target="_blank" style="color: rgb(17, 85, 204);">Twitter</a></span></span><span style="color: rgb(31, 73, 125);"> | </span><a href="http://www.linkedin.com/in/dieswaytoofast" target="_blank" style="color: rgb(17, 85, 204);">LinkedIn</a></div></span></div></div></div></b></span></font></div></div></div></div> <br><p style="color:#A0A0A8;">On February 17, 2014 at 4:40:22 PM, Miles Fidelman (<a href="mailto://mfidelman@meetinghouse.net">mfidelman@meetinghouse.net</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div><div>Mahesh,<br><br>Mahesh Paolini-Subramanya wrote:<br>> “Large number of processes with very long persistence”<br>><br>> You *will* run into GC issues here, and of all kinds<br>> - design artifacts (“hmm, the number of lists that I manipulate <br>> increases relentlessly…”)<br>> - misunderstanding (“But I passed the binary on, without <br>> manipulating it at all!”)<br>> - Bugs (Fred has a great writeup on this somewhere)<br><br>Very good points - though to a degree they sound more like dependency <br>hell than traditional garbage collection to reclaim memory.<br><br>Given the document-oriented view, I'm viewing garbage collection more in <br>the sense of filing and archiving - the same way that paper documents <br>migrate to filerooms then to archives; or email and computer files <br>simply get buried deeper and deeper in one's file system; sometimes you <br>buy a larger drive; sometimes stuff migrates to off-site backup - but <br>you generally don't throw stuff away (though when working on <br>multi-author documents, one always comes back to how many intermediate <br>copies to retain "for the record" after the final version goes to print).<br><br>In one sense, this ends up looking a lot like managing a git repository <br>- more and more versions and branches accumulate, and so forth. And <br>once starts thinking about storing only change logs.<br><br>This is also what motivates my question about how to handle older, <br>largely inactive processes. It's one thing to bury a file deeper and <br>deeper in a file system - and still be able to find and access it (and <br>these days, search for it). It's another to think about migrating an <br>actor from RAM to disk, in a way that retains its ability to respond to <br>the infrequent message.<br><br>The other area I worry about is exponential growth in network traffic <br>and cpu cycles - assuming that a lot of documents will never completely <br>"die" - maybe an update will come in once week, or once a month, or <br>they'll get searched every once in a while - as the number of processes <br>increases, the amount of traffic will as well.<br><br>> Just keep in mind that in the end, you will almost certainly end up <br>> doing some form of manual GC activities. Again, the Heroku gang can <br>> probably provide a whole bunch of pointers on this…<br>><br><br>Can you say a bit more about what it is about Heroku that I should be <br>looking at? At first glance, it seems like a very different environment <br>than what we're talking about here (or are you thinking about manual <br>housekeeping for the virtual environment?).<br><br>And.. re. "Bugs (Fred has a great writeup on this somewhere)" - Fred <br>who? (Maybe I can find it by googling!)<br><br>Thanks Very Much,<br><br>Miles<br><br>-- <br>In theory, there is no difference between theory and practice.<br>In practice, there is. .... Yogi Berra<br><br>_______________________________________________<br>erlang-questions mailing list<br>erlang-questions@erlang.org<br>http://erlang.org/mailman/listinfo/erlang-questions<br></div></div></span></blockquote></body></html>