[erlang-questions] Flush "stdout" buffer

Michael Truog <>
Sun Sep 3 08:30:35 CEST 2017


On 09/02/2017 09:22 PM, Per Hedeland wrote:
> On 2017-09-02 19:15, Michael Truog wrote:
>> On 09/02/2017 04:08 AM, Per Hedeland wrote:
>>> On 2017-09-02 10:07, Michael Truog wrote:
>>>> On 09/01/2017 04:58 AM, Frank Muller wrote:
>>>>> Can someone help on this please?
>>>>>
>>>> If you use stdout as an Erlang port like:
>>>>
>>>> STDOUT =  erlang:open_port({fd, 0, 1}, [out, {line, 256}]),
>>>> erlang:port_command(STDOUT, "unbuffered").
>>>
>>> Can you actually observe a difference *in buffering* between doing that
>>> and just
>>>
>>> io:format("unbuffered").
>>
>> Yes, it really does matter.
>
> Unfortunately you didn't answer my question.
>
>> You will notice that the io module doesn't have any flush function,
>
> Yes, of course it doesn't have any flush function, since it doesn't do
> any buffering.

When you call a function in the io module, what happens?  An io:format/1 function call sends the data in an Erlang message and that Erlang message goes into the group leader's Erlang process message queue, to get received when the Erlang process is ready for it.  That use of the Erlang process message queue is a form of buffering, ignoring everything that happens after that in the group leader Erlang process.

>
>> so you are unable to force any buffering in the Erlang VM to flush.
>
> This gets us into the realm of philosophy - is it possible to be unable
> to flush buffering that doesn't exist?

Refer to the explanation above.

>
>> However, if you really care about stdout in unbuffered mode, you need to use the Erlang port as
>> described above.
>
> Please provide *some* justification for that claim. At least a way to
> observe that buffering is actually happening - ideally a pointer to the
> specific piece of code that implements it.
>
> I don't care much what you do in your code, but I strongly oppose that
> you lead "newbies" to believe that they have to resort to ugly low-level
> stuff like the above in order to achieve functionality that is actually
> the default.

To allow output to be unbuffered, you need to bypass buffers.  If the approach isn't simple enough, that means there is room for improvement in the Erlang programming language.

>
>> That is similar to using fcntl to set file descriptor 1 with O_NONBLOCK.
>
> No, neither that code nor making C stdout unbuffered is even remotely
> similar to making file descriptor 1 non-blocking.
>
>> Programming languages normally have a way of setting stdout (and stderr, though stderr is really suppose to always be in unbuffered mode) in an unbuffered mode so all the output is available as quickly as possible.
>
> Yes, programming languages that have C stdio or a corresponding
> buffering mechanism normally have a way of turning it off. Erlang
> doesn't have such a mechanism (for non-disk I/O).
>
> [snip irrelevant elaboration on C/C++ programming]
>
>> In Erlang source code we can't really put the io server into an unbuffered mode, and there is no reason to really, because it is simpler to bypass the io server by making an Erlang port, as shown above.
>
> No - there is no reason to do it, because it isn't needed.
>
> Btw, this doesn't mean that Erlang "standard_io" is somehow "higher
> level" than C stdio, it's rather the opposite - but it's simpler to use,
> since you don't need to care about tweaking the buffering with setvbuf()
> and/or fflush() like you do in C.
>
> And finally, yet another demonstration of the lack of buffering in
> Erlang "standard_io" - a progress bar!:-)
>
> Eshell V8.3.5.1  (abort with ^G)
> 1> F = fun (0,_) -> ok; (N,G) -> io:format("*"),timer:sleep(1000),G(N-1,G) end.
> #Fun<erl_eval.12.118419387>
> 2> F(20,F).
>
To have a fast responsive progress bar, you need unbuffered output that won't get stuck on other use of io, allowing unimpeded use of the \b character that goes directly to the terminal without special terminal settings.  If you don't understand this, I have an example at https://github.com/okeuday/reltool_util/blob/b80cf8ecc423a108402529272ffe00f018bcf5ca/scope#L873-L944 .

Best Regards,
Michael



More information about the erlang-questions mailing list