Are NIF resources meant to be destroyed immediately on GC?

[Sean Hinde]

That's an interesting API function thanks for the pointer.

It's always possible that users can hang onto resources. Indeed I was once told a major telco had decided against using Erlang in some in house team because their top C++ programmer had tried it and found a way to leave a socket connection open once it had "gone out of scope".

Sean

> On 25 Nov 2020, at 17:05, Sverker Eriksson <sverker.eriksson@REDACTED> wrote:
> 
> I fully agree with Jesper about not relying on GC for resource reclamation. An additional risk is that the resource does not get GC’ed at all if its reference has “leaked” away onto the heap of another process, like for example a logger or an io-server process.
>  
> For the error case when a resource owning process crashes, a good alternative can be to use https://erlang.org/doc/man/erl_nif.html#enif_monitor_process <https://erlang.org/doc/man/erl_nif.html#enif_monitor_process> and reclaim the resource in the down callback.
>  
> /Sverker, Erlang/OTP
>  
> From: Sean Hinde <sean.hinde@REDACTED> 
> Sent: den 25 november 2020 11:36
> To: Jesper Louis Andersen <jesper.louis.andersen@REDACTED>
> Cc: Sverker Eriksson <sverker.eriksson@REDACTED>; Erlang Questions <erlang-questions@REDACTED>
> Subject: Re: Are NIF resources meant to be destroyed immediately on GC?
>  
> I agree. My use case for this was building a nif for the oracle oci bindings.
>  
> In the happy path users should call the provided db_close function which would flush outstanding transactions and close everything down in the dirty nif.
>  
> The problematic case is where the user's process holding the connection crashes. In that case we need the finaliser as the "secondary" resource reclamation path. If that path cannot safely close the db handle for fear of screwing up the emulator it's a problem.
>  
> The solution today is to have an interim process to fault isolate the nif holding process from the user process. Which adds overhead.
>  
> Sean
> 
> 
> On 25 Nov 2020, at 11:23, Jesper Louis Andersen <jesper.louis.andersen@REDACTED <mailto:jesper.louis.andersen@REDACTED>> wrote:
>  
> As a general rule: never use finalizers for the primary resource reclamation path. You don't know when they'll run. You don't know what changes are made to a GC in the future, that alters the rule set slightly. Use them as a safety measure if something goes out of scope. Resources such as DB connections can be very limited in scope (less than 50 for a typical non pg_bouncer postgres setup). So you have to manage them quite explicitly. Memory is far more abundant, and it is also fungible to a large extent, so you just need a block somewhere.
>  
> This is true in *any* GC language I've worked with over the years. There may come something along which completely changes the landscape, but it hasn't happened yet. It's more in the other direction (see e.g., Ephemerons[0]) that things are progressing.
>  
> [0] https://en.wikipedia.org/wiki/Ephemeron <https://en.wikipedia.org/wiki/Ephemeron>
>  
> On Wed, Nov 25, 2020 at 10:54 AM Sean Hinde <sean.hinde@REDACTED <mailto:sean.hinde@REDACTED>> wrote:
>  
> This a really nice change. It opens the door to resource destructors being safe for closing external resources like database handles that might take a while.
>  
> Super cool!
>  
> Sorry to disappoint, but no.
> By ”scheduled” I just meant “put in run queue to be executed at a later time”. Destructors are still executed by a normal scheduler threads and not dirty schedulers.
>  
> Ah well! I still think it would be pretty handy. It would make that kind of code much easier to get right, safer, and potentially more efficient.
>  
> Sean
>  
> 
>  
> -- 
> J.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20201125/a4fe11e6/attachment.htm>