[erlang-questions] Erlang shows its slow face!

[Edmond Begumisa]

Hello Frédéric,

> Spawning parallel processes is, in my opinion, a false ideal when it  
> comes to algorithms like the ones presented here. In a micro-benchmark  
> or in software that is oriented on doing only this task, multiprocessing  
> might gain you points. In a loaded production system where all the  
> processors are likely already busy, I wouldn't be surprised to see no  
> big gain by using multiple processes, and if any, the gain would likely  
> be very hard to predict.
>
> I think it's better to aim for the scaling kind of concurrency, not the  
> speed-up kind. If you use processes for truly concurrent tasks (rather  
> than attempting speedups of a single call), You'll benefit from  
> concurrency on a higher level on a server than from each function  
> individually.
>
> This has to be pretty application specific, though.

Hmmm... I see what you mean.

I look at it from an interaction perspective. If the application in  
question is an interactive one, I really hate to see it struggling with a  
problem for a long period of time with one of my cores saying "help!"  
while the other one just sits there, and more importantly, while *I* just  
sit there -- regardless of the algorithm it is working. (Any other  
Photoshop users been annoyed by this?)

For a non-interactive/server application, I can see why you might not want  
to do this. But then again, if the muscle is there, and interactive  
clients are waiting... tough call. I guess it's very application specific  
as you said.

One interesting thing I found from Garcia's results is that for this  
particular problem, crunching it using many processes on a single core was  
basically the same as doing it with one process on a single core. I was  
pleasantly surprised by that and didn't expect it.

- Edmond -



On Wed, 17 Nov 2010 02:21:00 +1100, Frédéric Trottier-Hébert  
<fred.hebert@REDACTED> wrote:

>
> On 2010-11-16, at 09:51 AM, Evans, Matthew wrote:
>
>> Interesting thread...I think we can all take the following from this:
>>
>> 1) Write your solution in the "Erlang way", using Erlang's strengths  
>> such as spawning parallel processes, also HIPE and NIFS where  
>> appropriate.
> Spawning parallel processes is, in my opinion, a false ideal when it  
> comes to algorithms like the ones presented here. In a micro-benchmark  
> or in software that is oriented on doing only this task, multiprocessing  
> might gain you points. In a loaded production system where all the  
> processors are likely already busy, I wouldn't be surprised to see no  
> big gain by using multiple processes, and if any, the gain would likely  
> be very hard to predict.
>
> I think it's better to aim for the scaling kind of concurrency, not the  
> speed-up kind. If you use processes for truly concurrent tasks (rather  
> than attempting speedups of a single call), You'll benefit from  
> concurrency on a higher level on a server than from each function  
> individually.
>
> This has to be pretty application specific, though.
>
>> 2) Write your solution efficiently - true in all languages, but  
>> especially in Erlang where you could end up doing needless recursion.
>>
>> 3) Erlang isn't always the best language for all problems. Neither is  
>> C, C++, C#, Java or any other language. Choose the language, or a mix  
>> of languages, that are the best fit your problem.
>>
>> What I would like to add is that we are trying to get Erlang accepted  
>> in a very C/C++ centric development environment. Although time and time  
>> again Erlang has shown itself a superior language and development  
>> environment for our domain, there is still a big problem in people  
>> accepting it. One of the main stumbling blocks is it is perceived to be  
>> "slow". People always say "it won't be faster than a C/C++ solution",  
>> or "a language written in a VM will always be slow [rolleyes]".
>>
>> We can argue until we are blue in the face that it will be as good or  
>> better when you throw multi-cores in the mix. That there are other  
>> advantages including shorter development time, better debugging,  
>> redundancy, failure detection etc. But overcoming that initial "speed"  
>> bias is a tough sell. I'm not saying there is an answer to this, and I  
>> know that the development team is doing their best to make Erlang  
>> faster, but we mustn't forget that for many, the "perceived" slowness  
>> is one factor that prevents them developing in Erlang.
>>
>
> I think the speed obsession has to be remnants of an education always  
> based on time efficiency, especially for programmers coming from the era  
> where processors were really slow. Whether this explanation is right or  
> not, I don't think it is easily possible to convince someone that 1. his  
> priorities are not the right one and 2. the language he uses isn't the  
> best choice for the priorities he has.
>
> We should consider that the burden of proof is ours, not theirs. Any  
> unsubstantiated claim is usually met with skepticism and possibly  
> violence. This is completely normal and natural, in my opinion. The best  
> way to convince someone is with irrefutable proof, which might be why  
> the most common suggested way to get a team to accept Erlang is to  
> provide them with an implementation or success stories. And maybe Erlang  
> is no better than what they're using right now, in which case it would  
> be a waste of money to train everyone to work in a new environment with  
> little benefit in the long run.
>
> If people still are not accepting it even after valid proofs, then all  
> bets are off: maybe they don't feel like learning new technology, maybe  
> they don't believe that *they* can write efficient Erlang code (and then  
> you have to prove them they can), maybe they think it all comes down to  
> the same anyway (project success isn't language dependent? which could  
> be true in many cases), etc. I'm not sure much can be done except  
> leading by example, in any case.
>
>> /Rant over
>>
>> Matt
>>
>
>
> --
> Fred Hébert
> http://www.erlang-solutions.com
>
>
>
>


-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/