[erlang-bugs] efile_drv & async thread key

[Patrik Nyblom]

Hi Rick!

On 08/14/2013 02:21 AM, Rick Reed wrote:
> I assume the reason for keying the file requests is to prevent a 
> single port from
> soaking up all the async threads?
Yes, and it's also important that requests for the same file 
"descriptor" end up in she same async queue. So we need to store a fixed 
key in the file descriptor structure.

I think I will hash the pointer to create the key, not just shift away 
the "zero-bits", you never know which icky patterns an allocator can 
create that will distribute the jobs unevenly. The key will only be 
calculated upon opening, so there will be minimal performance hit due to 
the more complicated calculations.

Thanks for reporting - this could cause severe performance issues in 
applications!

Cheers,
Patrik
>
> Rr
>
>
> On Tue, Aug 13, 2013 at 4:52 AM, Lukas Larsson <lukas@REDACTED 
> <mailto:lukas@REDACTED>> wrote:
>
>     And there it is, conclusive proof that I should not be debugging
>     Rickard's code before lunch.
>
>     Found the issue, will create a fix for it. As a workaround for
>     R16B you can use a prime number as the number of async threads :)
>
>     Lukas
>
>
>     On 13/08/13 10:05, Lukas Larsson wrote:
>>     Sigh, apparently I spoke too soon.
>>
>>     I remembered incorrectly about the change. It was in R16B that
>>     ErlDrvPort became a ptr and it was an id before R16B. Anyways, it
>>     is odd that the ptr is 8 bit aligned on you system. On mine
>>     (Ubuntu 13.04, x86_64) the ptrs are not aligned and the load is
>>     nicely distributed among async threads. If I remember correctly
>>     you are using FreeBSD on x86_64? I'll check if I can reproduce
>>     the behavior you are seeing on our FreeBSD machine.
>>
>>     Lukas
>>
>>     On 13/08/13 09:40, Lukas Larsson wrote:
>>>     Hello Rick!
>>>
>>>     Which version of Erlang are you using? From R16B (I think), the
>>>     ErlDrvPort datatype no longer is a pointer to the port struct.
>>>     Instead it is the slot id into the port table and those ids
>>>     should contain all values. I did a quick test on my computer
>>>     running the latest on maint on github and seem to get a full
>>>     spread over all async threads.
>>>
>>>     Lukas
>>>
>>>     On 13/08/13 05:40, Rick Reed wrote:
>>>>     It looks to me as though there's a bit of a problem in the way
>>>>     efile_drv.c generates the
>>>>     key that's used to select an async driver queue.  It uses the
>>>>     address of the port which
>>>>     on our system is 8-byte aligned.  Meanwhile, erl_async.c does a
>>>>     simple mod operation
>>>>     with the number of async threads, so the number of threads that
>>>>     can actually be used
>>>>     by file operations is 1/8th of the number configured.  I
>>>>     suspect this isn't intended.
>>>>
>>>>     Rr
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     erlang-bugs mailing list
>>>>     erlang-bugs@REDACTED  <mailto:erlang-bugs@REDACTED>
>>>>     http://erlang.org/mailman/listinfo/erlang-bugs
>>>
>>>
>>>
>>>     _______________________________________________
>>>     erlang-bugs mailing list
>>>     erlang-bugs@REDACTED  <mailto:erlang-bugs@REDACTED>
>>>     http://erlang.org/mailman/listinfo/erlang-bugs
>>
>>
>>
>>     _______________________________________________
>>     erlang-bugs mailing list
>>     erlang-bugs@REDACTED  <mailto:erlang-bugs@REDACTED>
>>     http://erlang.org/mailman/listinfo/erlang-bugs
>
>
>
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-bugs/attachments/20130814/d1fc578d/attachment.htm>