pcre, bifs, drivers and ports

[Sean Hinde]

Hi Ernie,

On 1 Aug 2006, at 14:04, Ernie Makris wrote:

> Hi Sean,
>
> Is that one of the drawbacks to a bif implementation? Namely, the bif
> call stops the entire emulator? Even with the
> SMP version of the emulator?

I'm not that familiar with the SMP version of the emulator.. yet..  
but I would think that a blocking bif in one member of the SMP thread  
pool would only block that thread. Then it depends of other stuff  
like for example I understand that I/O uses one big lock in the  
current SMP implementaion.

That is why I wrote it would be interesting to test :-)

> If this is the case, then I can definitely see why a driver based pcre
> should be preferable for a potentially long
> running regex computation.

I agree. Anytime the overhead of the driver call is small compared to  
the work to be done then there seems little point in trying to  
optimise away the overhead. Let's see if Mats' work ends up giving  
some nice results - if he is happy to share.

Sean

>
> Thanks
> Ernie
> Sean Hinde wrote:
>> Hi Ernie,
>>
>> On 31 Jul 2006, at 16:21, Ernie Makris wrote:
>>
>>> Hi Sean
>>>
>>>>
>>>> If you add this feature by hacking the otp release and adding  
>>>> bifs you
>>>> would create all the long term issues of having a patched OTP  
>>>> release.
>>>>
>>> Thats ok, I already patch my releases in order to fix a linux to_erl
>>> bug, and to allow the file driver to open /dev/urandom. The only  
>>> pain is
>>> verifying the patch on a new release. But I know the code well  
>>> enough
>>> that I can do it by hand each time.
>>
>> Where I work we already have a quite heavily patched OTP release so I
>> know well the pain. For me the hassle of yet more patches would  
>> almost
>> certainly outweigh some small speed benefit..
>>
>>>> You would unlikely to get adopters of any open source version you
>>>> released for much the same reason.
>>>>
>>> I would still put it out there, if anyone wanted to use it, they  
>>> would
>>> be welcome.
>>
>> And I guess if they liked it enough they could add a driver  
>> option :-)
>>
>>>> OTOH it is not too hard to create a linked in driver that could
>>>> alternatively be compiled as a port program. That way you give  
>>>> users a
>>>> choice between speed of a driver (almost the same as a BIF), or  
>>>> fault
>>>> isolation.
>>>>
>>> I agree, but I also wanted to see how easy/difficult it would be  
>>> to add
>>> a few new bifs.
>>
>> One more thing to think on - assuming that pcre itself is thread  
>> safe,
>> adding it as a driver would make multithreading quite  
>> straightforward.
>> Mats' comment about limiting length of REs does not really cut it  
>> IMO.
>> Blocking the whole emulator during a long regexp calculation rarely
>> sounds like the right solution for typical Erlang apps.
>>
>> But. It would be most fascinating to compare real world
>> characteristics of:
>>
>> 1. BIF pcre
>> 2. Driver pcre,
>> 3. BIF pcre in SMT erlang.
>>
>> Sean
>>
>>
>>
>>