Simple use of jobs application failing
Ulf Wiger
ulf@REDACTED
Mon Nov 15 07:16:50 CET 2021
BTW, you can of course use the shorthand notation for this as well.
7> jobs:add_queue(otto, [{standard_counter,3},{max_size,3}]).
ok
8> [spawn(fun() -> Res = jobs:ask(otto), io:fwrite("~p --> ~p: ~p~n", [Res,
time(), self()]), timer:sleep(3000), exit(normal) end) || _ <-
lists:seq(1,10)].
{ok,{#Ref<0.4079473062.3785097217.122736>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,39}: <0.4238.0>
{ok,{#Ref<0.4079473062.3785097217.122740>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,39}: <0.4239.0>
{ok,{#Ref<0.4079473062.3785097217.122742>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,39}: <0.4240.0>
{error,rejected} --> {7,12,39}: <0.4244.0>
{error,rejected} --> {7,12,39}: <0.4245.0>
{error,rejected} --> {7,12,39}: <0.4246.0>
{error,rejected} --> {7,12,39}: <0.4247.0>
[<0.4238.0>,<0.4239.0>,<0.4240.0>,<0.4241.0>,<0.4242.0>,
<0.4243.0>,<0.4244.0>,<0.4245.0>,<0.4246.0>,<0.4247.0>]
{ok,{#Ref<0.4079473062.3785097217.122752>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,42}: <0.4241.0>
{ok,{#Ref<0.4079473062.3785097217.122753>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,42}: <0.4242.0>
{ok,{#Ref<0.4079473062.3785097217.122754>,
[{counters,[{{counter,otto,1},1}]}]}} --> {7,12,42}: <0.4243.0>
And you can limit the queue also based on time - and both size and time at
the same time.
9> jobs:add_queue(jane,
[{standard_counter,3},{max_size,3},{max_time,2000}]).
ok
10> [spawn(fun() -> Res = jobs:ask(jane), io:fwrite("~p --> ~p: ~p~n",
[Res, time(), self()]), timer:sleep(3000), exit(normal) end) || _ <-
lists:seq(1,10)].
{ok,{#Ref<0.4079473062.3785097217.122770>,
[{counters,[{{counter,jane,1},1}]}]}} --> {7,14,46}: <0.4262.0>
{ok,{#Ref<0.4079473062.3785097217.122774>,
[{counters,[{{counter,jane,1},1}]}]}} --> {7,14,46}: <0.4263.0>
{ok,{#Ref<0.4079473062.3785097217.122776>,
[{counters,[{{counter,jane,1},1}]}]}} --> {7,14,46}: <0.4264.0>
{error,rejected} --> {7,14,46}: <0.4268.0>
{error,rejected} --> {7,14,46}: <0.4269.0>
{error,rejected} --> {7,14,46}: <0.4270.0>
{error,rejected} --> {7,14,46}: <0.4271.0>
[<0.4262.0>,<0.4263.0>,<0.4264.0>,<0.4265.0>,<0.4266.0>,
<0.4267.0>,<0.4268.0>,<0.4269.0>,<0.4270.0>,<0.4271.0>]
{error,timeout} --> {7,14,48}: <0.4267.0>
{error,timeout} --> {7,14,48}: <0.4266.0>
{error,timeout} --> {7,14,48}: <0.4265.0>
BR,
Ulf W
On Mon, Nov 15, 2021 at 7:11 AM Ulf Wiger <ulf@REDACTED> wrote:
> Well, you can.
>
> 4> jobs:add_queue(becka,
> [{regulators,[{counter,[{limit,3}]}]},{max_size,3}]).
> ok
> 5> [spawn(fun() -> Res = jobs:ask(becka), io:fwrite("~p --> ~p: ~p~n",
> [Res, time(), self()]), timer:sleep(3000), exit(normal) end) || _ <-
> lists:seq(1,10)].
> {ok,{#Ref<0.4079473062.3785097227.122199>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,46}: <0.4192.0>
> {ok,{#Ref<0.4079473062.3785097227.122196>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,46}: <0.4191.0>
> {ok,{#Ref<0.4079473062.3785097227.122202>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,46}: <0.4193.0>
> {error,rejected} --> {7,6,46}: <0.4197.0>
> {error,rejected} --> {7,6,46}: <0.4198.0>
> {error,rejected} --> {7,6,46}: <0.4199.0>
> {error,rejected} --> {7,6,46}: <0.4200.0>
> [<0.4191.0>,<0.4192.0>,<0.4193.0>,<0.4194.0>,<0.4195.0>,
> <0.4196.0>,<0.4197.0>,<0.4198.0>,<0.4199.0>,<0.4200.0>]
> {ok,{#Ref<0.4079473062.3785097217.122718>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,49}: <0.4194.0>
> {ok,{#Ref<0.4079473062.3785097217.122720>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,49}: <0.4195.0>
> {ok,{#Ref<0.4079473062.3785097217.122721>,
> [{counters,[{{counter,becka,1},1}]}]}} --> {7,6,49}: <0.4196.0>
>
> So, basically, the first three requests were served and removed from the
> queue. Meanwhile, a bunch of requests were rejected due to the size limit.
> Eventually, a few new requests were able to enter. After 3 seconds (when
> the first 3 workers woke up and finished), they were able to run. Now the
> queue is empty again.
>
> BR,
> UIf W
>
> On Sun, Nov 14, 2021 at 11:13 PM bengt <cean.ebengt@REDACTED> wrote:
>
>> Thank you very much. With this explanation I should get my application
>> properly limited.
>>
>> One thing though, is it not possible to get an error from jobs:ask/1 ?
>> Like {error, Reason}
>> when the queue is full? That is, jobs:done/1 have not been called for the
>> previous 3 {ok, X}.
>>
>> bengt
>>
>> On 14 Nov 2021, at 19:55, Ulf Wiger <ulf@REDACTED> wrote:
>>
>> In jobs, `max_size` has to do with how large the queue is allowed to get.
>> What you appear to be looking for is `counter`.
>>
>> Example:
>> Eshell V10.7.2.8 (abort with ^G)
>> 1> application:ensure_all_started(jobs).
>> {ok,[jobs]}
>> 2> jobs:add_queue(kalle, [{standard_counter,3}]).
>> ok
>> 3> [spawn(fun() -> jobs:ask(kalle), io:fwrite("--> ~p: ~p~n", [time(),
>> self()]), timer:sleep(3000), exit(normal) end) || _ <- lists:seq(1,10)].
>> --> {19,53,29}: <0.128.0>
>> --> {19,53,29}: <0.129.0>
>> --> {19,53,29}: <0.130.0>
>> [<0.128.0>,<0.129.0>,<0.130.0>,<0.131.0>,<0.132.0>,
>> <0.133.0>,<0.134.0>,<0.135.0>,<0.136.0>,<0.137.0>]
>> --> {19,53,32}: <0.131.0>
>> --> {19,53,32}: <0.133.0>
>> --> {19,53,32}: <0.132.0>
>> --> {19,53,35}: <0.134.0>
>> --> {19,53,35}: <0.135.0>
>> --> {19,53,35}: <0.136.0>
>> --> {19,53,38}: <0.137.0>
>>
>> From the docs:
>> "{standard_counter, C} - equivalent to [{regulators,[{counter,[{limit,C},
>> {modifiers,[{cpu,10},{memory,10}]}]}]}]"
>>
>> (Usually, one doesn't specify the modifiers)
>>
>> BR,
>> Ulf
>>
>> On Sat, Nov 13, 2021 at 6:12 PM bengt <cean.ebengt@REDACTED> wrote:
>>
>>> Greetings,
>>>
>>> I would like to allow at most 3 processes to call a gen_server at the
>>> same time. I could check the message queue length, but why not use jobs (
>>> https://github.com/uwiger/jobs) ?
>>>
>>> Jobs is already in use by something I depend on, so it should be
>>> working. I test from the (iex) shell. As can be seen below I am making a
>>> mistake somewhere. Probably config, so how should I configure to only get 3
>>> of the 5 jobs:ask/1 to be approved and queued? Or I have misunderstood the
>>> use, and an explanation of how to achieve the 3 caller limitation would be
>>> appreciated.
>>>
>>> :jobs.add_queue(:kalle, [{:max_size, 3}])
>>> for _ <- [1,2,3,4,5], do: :jobs.ask(:kalle)
>>> IO.inspect :jobs.queue_info(:kalle)
>>>
>>> name: :kalle,
>>> max_size: 3,
>>> approved: 5,
>>> queued: 5,
>>> latest_dispatch: 378734152926266,
>>> mod: :jobs_queue,
>>> type: :fifo,
>>> group: :undefined,
>>> regulators: [],
>>> max_time: :undefined,
>>> check_interval: :infinity,
>>> oldest_job: :undefined,
>>> timer: :undefined,
>>> link_ref: :undefined,
>>> check_counter: 0,
>>> empty: false,
>>> depleted: false,
>>> waiters: [],
>>> stateful: :undefined,
>>> st: {:st, #Reference<0.2285429207.3584425985.10312>},
>>> producers: []
>>>
>>> Best Wishes,
>>> bengt
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20211115/2113fb4a/attachment.htm>
More information about the erlang-questions
mailing list