[erlang-questions] [ANN] Priority Queue Implementation

Thu Nov 10 08:45:36 CET 2011

I modified the example to extend it with queues and it does compare very well, being slightly faster.  I still believe it needs to be tested more, but the implementation becomes simpler and you don't need the static priority limitations, which is nice.  The link is:
https://github.com/okeuday/pqueue/blob/master/src/pqueue2.erl

with erlbench results here ((with R14B02, without HiPE) on an AMD Phenom 9950 Quad-Core (64 bit) running Linux 2.6.32-23-generic (Ubuntu)):
TEST run_priority_queue
N == 1000000 (10 runs)
              pqueue get:   481774.3 µs (  1.4), set:   525589.1 µs (  1.0)
             pqueue2 get:   332711.2 µs (  1.0), set:   680209.0 µs (  1.3)
      priority_queue get:   362588.9 µs (  1.1), set:  1443674.2 µs (  2.7)

On 11/09/2011 08:58 AM, Michael Truog wrote:
> I think the skew heap I have needs some work, because it seems to come only from Okasaki's code example, so it doesn't take into consideration his suggestions/exercises.  So, only insert is O(1), and the min would need to be stored separately to get O(1) instead of O(log(N)). He had a suggestion for making the merge of two heaps O(1), but I wasn't as concerned about that operation.  It seems hard to get an "out" operation that is O(1) amortized, that is removing the min from the heap (hopefully O(log(N)) worst case).  I will look at testing a heap implementation to see how it might work out.  Thanks for the information.
>
> On 11/09/2011 01:00 AM, Ulf Wiger wrote:
>>
>> Yeah, obviously, mine was just a sketch, thrown down as an executable comment and optimized for brevity. :)
>>
>> (Although I'm not convinced, from reading, that Michael's implementation is faster than mine. Anyone who cares deeply enough could of course measure. I am currently not shopping for a faster priority queue, so I will pass on that.)
>>
>> As an aside, it was a simple skew heap exercise, presented by Chris Okasaki, that made me invite Quviq to Ericsson for the first Erlang QuickCheck pilots. 
>>
>> The task was to reverse-engineer the insertion order of a particular skew heap. John Hughes solved it with a "brute force approach", using QuickCheck to test his assumptions. Watching him do exploratory hacking with QuickCheck was so much fun that, once he ported QuickCheck to Erlang, I had to try to find out if it could be put to use in a commercial project.
>>
>> Unfortunately - or fortunately - for Quviq, the only candidate for a useful pilot was stateful, and QuickCheck had no support for that. For lesser minds, that might have been a problem, but John and Thomas quickly invented the statem model. :)
>>
>> BR,
>> Ulf W
>>
>> On 9 Nov 2011, at 09:45, Zabrane Mickael wrote:
>>
>>> Hi Ulf,
>>>
>>> Michael Truog already has a SkewBinHeap impelmentation here:
>>> https://github.com/okeuday/skewbinheap
>>>
>>> Regards,
>>> Zabrane
>>>
>>> On Nov 9, 2011, at 9:42 AM, Ulf Wiger wrote:
>>>
>>>> I'm partial to skew heaps, mainly because they are so elegant.
>>>>
>>>> http://www.cse.yorku.ca/~andy/courses/4101/lecture-notes/LN5.pdf <http://www.cse.yorku.ca/%7Eandy/courses/4101/lecture-notes/LN5.pdf>
>>>>
>>>> Something like this (although I've done only basic testing):
>>>>
>>>> -module(skew).
>>>> -export([new/0, in/2, out/1]).
>>>>
>>>> new() ->
>>>>     [].
>>>>
>>>> in(X, Heap) ->
>>>>     merge({X,[],[]}, Heap).
>>>>
>>>> out([]) ->
>>>>     error;
>>>> out({X, L, R}) ->
>>>>     {X, merge(L, R)}.
>>>>
>>>> merge({P0,Pl,Pr}, {Q0,_,_} = Q) when P0 < Q0 ->
>>>>     {P0, Pr, merge(Pl,Q)};
>>>> merge({P0,_,_} = P, {Q0,Ql,Qr}) when P0 > Q0 ->
>>>>     {Q0, Qr, merge(Ql,P)};
>>>> merge({P0,Pl,Pr} = P,{P0,Ql,Qr}) ->   % equal roots
>>>>     merge(P, merge(merge(Pl,Pr), merge(Ql,Qr)));
>>>> merge([], Q) -> Q;
>>>> merge(P, []) -> P.
>>>>
>>>> The cost is amortized O(log N) for in/2 and out/1. For peeking at the min, it's O(1).
>>>>
>>>> BR,
>>>> Ulf W
>>>>
>>>> On 9 Nov 2011, at 04:33, Michael Truog wrote:
>>>>
>>>>> I was looking at Erlang priority queue implementations and the Riak/RabbitMQ one seemed a bit slow.  I have a different implementation with the same API here: https://github.com/okeuday/pqueue/blob/master/src/pqueue.erl
>>>>>
>>>>> The results from my test are here: http://okeuday.livejournal.com/19187.html
>>>>>
>>>>> The implementation has "in" operations that are roughly 3 times faster (300%), however, the "out" operation became roughly 30% slower.  So, as long as the priority queue is storing a decent amount of items, this data structure should provide better speed.  The implementation is limited to a specific priority range: -20 (high) to 20 (low).
>>>>> _______________________________________________
>>>>> erlang-questions mailing list
>>>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>>
>>>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
>>>> http://erlang-solutions.com <http://erlang-solutions.com/>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> erlang-questions mailing list
>>>> erlang-questions@REDACTED <mailto:erlang-questions@REDACTED>
>>>> http://erlang.org/mailman/listinfo/erlang-questions
>>>
>>>
>>>
>>
>> Ulf Wiger, CTO, Erlang Solutions, Ltd.
>> http://erlang-solutions.com
>>
>>
>>
>
>
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@REDACTED
> http://erlang.org/mailman/listinfo/erlang-questions

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20111109/d100e69a/attachment.htm>