[erlang-questions] ei_send multithreaded problem

[lang er]

I googled another thread on TrapExit
http://forum.trapexit.org/viewtopic.php?p=23802&sid=af4d7236885d03b4cac285a5073ae371

In the last post, Bob, said


> Well, after much head scratching I've got it stable. Bad idea to send
> messages from inside a callback function, even very small ones like just
> {sync} to set off the process. It's still dog slow, it's been running a few
> minutes and there are over 5000 queued messages but at least it stays up
> long enough to find out why?
>

I don't know why he got the conclusion that it is a bad idea to send
messages from inside a callback function?
But I do inspect muy process, there's no unmatched message in queue.


BR
James

2007/3/1, lang er <erlangist@REDACTED>:
>
> Hi Eduardo,
>
> After the previus modification failed, I also modify code as what you
> give.This time the normal status last for more time than before,
> but eventually (about one hours later), I get this error again.
>
> The data transfered between Cnode and Erlang process is
> large(240k/second). I also run this program on a different machine, and use
> small
> data (4k/second), it just work fine for more than 24 hours(and I exit it
> manually).
>
>
> Thanks!
>
> BR
> James
> 2007/3/1, Eduardo Figoli (INS) <eduardo@REDACTED>:
> >
> >  Try
> >
> > >  EnterCriticalSection(&CriticalSection);
> >
> > > erlang_pid *channel_connection_pid = (erlang_pid *)dwUser;
> > >  ei_x_buff xbuf;
> > >  ei_x_new_with_version(&xbuf);
> > >  ei_x_encode_tuple_header(&xbuf,2);
> > >  ei_x_encode_atom(&xbuf,"data");
> > >  ei_x_encode_binary(&xbuf, pBuffer, dwBufSize);
> > >  ei_send(acceptedFd, channel_connection_pid, xbuf.buff,xbuf.index);
> > >  ei_x_free(&xbuf);
> > >  LeaveCriticalSection(&CriticalSection);
> >
> >
> >
> >
> > Regards Eduardo
> >
> >
> >  ------------------------------
> >
> > *De:* lang er [mailto:erlangist@REDACTED]
> > *Enviado el:* Jueves, 01 de Marzo de 2007 05:41 a.m.
> > *Para:* Eduardo Figoli (INS)
> > *CC:* erlang-questions@REDACTED
> > *Asunto:* Re: [erlang-questions] ei_send multithreaded problem
> >
> >
> >
> > Hi, Eduardo
> >
> >
> > I add a critical section before and after ei_send,now the code look like
> > this:
> >
> >
> > > erlang_pid *channel_connection_pid = (erlang_pid *)dwUser;
> > >  ei_x_buff xbuf;
> > >  ei_x_new_with_version(&xbuf);
> > >  ei_x_encode_tuple_header(&xbuf,2);
> > >  ei_x_encode_atom(&xbuf,"data");
> > >  ei_x_encode_binary(&xbuf, pBuffer, dwBufSize);
> > >  EnterCriticalSection(&CriticalSection);
> > >  ei_send(acceptedFd, channel_connection_pid, xbuf.buff,xbuf.index);
> > >  LeaveCriticalSection(&CriticalSection);
> > >  ei_x_free(&xbuf);
> > >
> >
> >
> > now the mormal running time is longer, but Erlang side still got "ERROR
> > REPORT= Got invalid data on distribution channel, offending packet is:"
> > error eventually.
> >
> >
> > I  skip first 112 of the error message<<112,131,104,3,97...>>, and use
> > binary_to_term,get term like:
> >
> >
> >
> > {2,'',<4761.49.0>}
> >
> >
> >
> > I search the maillisting again, and only get one post about this:
> >
> >
> >
> >
> > http://www.erlang.org/pipermail/erlang-questions/2004-October/013383.html
> >
> >
> >
> > It looks like this problem is solved by using mutex
> >
> > http://www.erlang.org/pipermail/erlang-questions/2004-October/013385.html
> >
> > But I still have this problem.
> >
> >
> >
> > Any help is appreciable!
> >
> >
> >
> > BR
> >
> >
> >
> > James
> >
> >
> >
> >
> > 2007/2/27, lang er <erlangist@REDACTED >:
> > >
> > > But in the erl_interface document, it said:
> > >
> > >
> > > > If you are using Erl Interface functions in a threaded application
> > based on POSIX threads or Solaris
> > > > threads, then Erl Interface needs access to some of the
> > synchronization facilities in your threads
> > > > package, and you will need to specify additional compiler flags in
> > order to indicate which of the
> > > > packages you are using. Define REENTRANT and either STHREADS or
> > PTHREADS. The default is to use
> > > > POSIX threads if REENTRANT is specified.
> > > > Note that both single threaded and default versions of the Erl
> > interface and Ei libraries are provided.
> > > > (The single threaded versions are named liberl interface st and
> > libei st). Whether the default
> > > > versions of the libraries have support for threads or not is
> > determined by if the platform in question has
> > > > support for POSIX or Solaris threads. To check this, have a look in
> > the eidefs.mk file in the
> > > > erl interface src directory.
> > >
> > >
> > > I checked the eidefs.mk file, its content is:
> > >
> > >
> > >
> > >
> > >
> > > # Have the ei and erl_interface libs been compiled for threads?
> > > EI_THREADS=true
> > >
> > > # Threads flags
> > > THR_DEFS=-DWIN32_THREADS -DUSE_DECLSPEC_THREAD
> > >
> > > # Threads libs
> > > THR_LIBS=
> > >
> > > #
> > ----------------------------------------------------------------------
> > >
> > >
> > > Doesn't it means I don't need use any synchronization facilities
> > myself?
> > >
> > > Thanks!
> > >
> > > James
> > >
> > > 2007/2/26, Eduardo Figoli (INS) <eduardo@REDACTED >:
> > >
> > > >
> > > >
> > > >
> > > >
> > > > Have you tried using a critical section whenever calling ei_send?
> > > >
> > > >
> > > >
> > > > Regards, Eduardo
> > > >
> > > >
> > > >
> > > > ________________________________
> >
> > > >
> > > > De: erlang-questions-bounces@REDACTED [mailto: erlang-questions-bounces@REDACTED
> > ] En nombre de lang er
> > > > Enviado el: Lunes, 26 de Febrero de 2007 01:05 p.m.
> > > > Para: erlang-questions@REDACTED
> > > > Asunto: [erlang-questions] ei_send multithreaded problem
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > I'm writing a c node using ei layer. This program is running under
> > Window 2003 enterprise edition sp1, and otp_win32_R11B-3. The following is a
> > fragment of my code:
> > > >
> > > >
> > > >
> > > > --------------------------------------------------------------------
> > > >
> > > >  erlang_pid *pid = (erlang_pid *)pUser;
> > > >  ei_x_buff xbuf;
> > > >  ei_x_new_with_version(&xbuf);
> > > >  ei_x_encode_tuple_header(&xbuf,2);
> > > >  ei_x_encode_atom(&xbuf,"data");
> > > >  ei_x_encode_binary(&xbuf, buffer, bufSize);
> > > >  ei_send(acceptedFd, pid, xbuf.buff,xbuf.index);
> > > >  ei_x_free(&xbuf);
> > > >
> > > >
> > ----------------------------------------------------------------------
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > It is called inside a C callback function and may be called by many
> > concurrent threads. pUser include a erlang_pid (correspond to a process in
> > Erlang side).When some data come from network, a third party library
> > will invoke this callback function, so ei_send pass these data to a
> > corresponding Erlang process.(This process is an Erlang equivalent to a 3rd
> > library thread. So if I start one process in Erlang side, the 3rd library
> > will use one thread. If I start more than one process in Erlang , the 3rd
> > library will use same number of threads.)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Then, If I start one process in Erlang side, everything is OK. But
> > If I start more than one process, the C node program will crash after some
> > random time. If I comment  out ei_send line in above code fragment,
> > everything is OK again.
> > > >
> > > >
> > > >
> > > > Do I miss anything to use ei_send in multithreaded environment?
> > > >
> > > >
> > > >
> > > > compiler: ms visual studio 2005
> > > >
> > > > debug mode use  ei_mdd.lib
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > James
> > > >
> > > >
> > >
> > >
> >
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20070301/108565a8/attachment.htm>