[erlang-questions] Logs and stdout [OFFTOPIC]

[Geoff Cant]

<fireside-chat-mode>
"12 Factor” (aka https://12factor.net) is a set of conventions to make linux applications easier to run in containers and to allow those containers to be managed and orchestrated uniformly. These conventions were established by one of Heroku’s co-founders, Adam Wiggins (hi Adam!), and are mostly still good ways to adapt linux applications to running on a Platform as a Service (Heroku/CloudFoundry/Deis/Flynn/Mesosphere/Kubernetes/…). 12 Factor is a little dated now - it’s last update was in 2012, but the ideas are still mostly relevant.


In the case of logging, the reasoning behind the stdout convention is that is was a relatively easy to implement (for the app developer), language/framework independent approach to getting a real-time ish stream of logs out of any application. The platform the application runs on can then handle log forwarding and distribution uniformly. This would have been hard to impossible to do if every application was allowed to choose its own method for emitting log events (syslog vs stdout vs files vs snmp traps vs probably lots of other things). Logs as files, in particular, would have to be a complicated convention (where do you write them, how many files do you write, how are they rotated, do they get truncated to avoid running out of space, what constitutes a complete message in a file if you’re trying to stream them, ...), and this complexity would have been development costs paid in every single application for the PaaS.

The corollary of this is that if you’re not running your application on a PaaS, then you’re not going to get much benefit from writing logs to stdout.


Aside: I worked on Heroku’s log pipeline systems for a while, and like to see this factor revisited. ‘\n' framed opaque-byte messages on stdout is a protocol due for improvement: you can’t write neatly formatted multi-line messages and have them pass through the log pipeline very easily (change the framing), it’s an async one-way protocol (there’s no facility for reliable transmission of messages, so you can’t use it for audit logs), it is sub-optimal transmission format for sending metrics (metrics can be aggregated during transport, and logs can’t). So let’s hope there are some updates to 12 factor in 2017. I have quibbles about configuration passed as environment variables too, but unix doesn’t offer a lot of good choices here.
</fireside-chat-mode>

-Geoff
ps: yeah - use lager. It’ll work both on a PaaS and your own box.

> On 28 Nov, 2016, at 18:05, Richard A. O'Keefe <ok@REDACTED> wrote:
> 
> 
> 
> On 25/11/16 11:34 PM, Alexander Petrovsky wrote:
>> - Hey, boy, did you know about 12 app factor (*oh my god*)?
> 
> "12-factor app" was not handed down on Mt Sinai;
> Jibril did not dictate it to the Prophet;
> Joseph Smith Jr never found it on the golden plates.
> Heck, even Raël never heard about it from the elohim.
> 
> "12-factor app" is OPINION.
> 
> Using lager, the vast bulk of your code doesn't know and
> doesn't care where the logs are going.  It *can* go to
> stdout if that's what you want.
> 
> On the other hand, if you need log rotation and such,
> and you only write to stdout, then you have to depend
> on something outside your application to do it.
> 
> stdout is where information is sent to die.  (:-)
> 
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions