<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 04/03/2013 10:56 AM, Max Lapshin
wrote:<br>
</div>
<blockquote
cite="mid:CAMxVRxAqT+zifRBKW4UGpFg6MFnU4Q_JoQoLMOE8TxTRsCgf1w@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<div dir="ltr">You are trying to solve another problem.
<div><br>
</div>
<div>Hardware can _always_ be bad. But whole system must not go
down if one disk is slow.</div>
<div><br>
</div>
<div>Currently, whole erlang is becoming useless if only one
hard drive from 16 doesn't respond.</div>
<div><br>
</div>
<div style="">This is the problem.</div>
</div>
</blockquote>
Wouldn't you rather need to have one async pool per device? Or the
option to bind certain files to specific async threads? The thing
with dynamic async threads is that the implementation relies on
requests for the same file descriptor being routed to the same
thread for all of the descriptors lifespan. If the async thread
hangs, other file descriptors already routed to that thread have to
wait. You would need to have the "good" mapping between threads and
devices from the beginning.<br>
<br>
Something like<br>
<br>
open_a_file(Name) -><br>
PoolID = get_pool_id_from_name(Name),<br>
{ok, FD} = <a class="moz-txt-link-freetext" href="file:open(Name">file:open(Name</a>,{async_pool_id, PoolId}),<br>
FD.<br>
<br>
and get_pool_id_from_name/1 will look at the file name, figure out
the device and return a pool id that is only used for that
particular device? It could even be a specific thread id - you could
always find out how many async threads you have, know how many
devices you have and round robin files between the threads that
should handle that particular device. In that case you would only
need to add an interface where you can specify the thread when
opening the file (and remenber which thread is connected to each
port). Sounds like a fairly feasible hack...<br>
<br>
On a side note, are you using sendfile? That might at least make the
ongoing connections run smooth while another device is blocked...
Not that it addresses the real problem, but just wondering if you
have tried using it - it might be good for performance in your
application.<br>
<br>
Another, less compelling, solution would be to route only the writes
through another process, so that you open a small "writer program"
and pump the data to it through the spawn driver. Not a general
solution either, but much simpler than to hack the file driver...<br>
<br>
The async thread pool leaves a lot to desire - we have planned to
replace it with something called "dirty schedulers" (internal
name...), which is in our roadmap for R17. In the end they should
solve a lot of the problems with the current implementation. Not
that that helps you now, but it makes me wonder if not a "dirty"
solution might be the right way to solve it for the moment, as
another solution is in the pipe...<br>
<blockquote
cite="mid:CAMxVRxAqT+zifRBKW4UGpFg6MFnU4Q_JoQoLMOE8TxTRsCgf1w@mail.gmail.com"
type="cite">
<div dir="ltr">
<div style=""><br>
</div>
<div style=""><br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
erlang-questions mailing list
<a class="moz-txt-link-abbreviated" href="mailto:erlang-questions@erlang.org">erlang-questions@erlang.org</a>
<a class="moz-txt-link-freetext" href="http://erlang.org/mailman/listinfo/erlang-questions">http://erlang.org/mailman/listinfo/erlang-questions</a>
</pre>
</blockquote>
Cheers,<br>
/Patrik<br>
</body>
</html>