[erlang-questions] Feedback on webmachine route

[Damien Krotkine]

Jesper Louis Andersen wrote:
>
> On Fri, Sep 25, 2015 at 6:03 PM, Damien Krotkine <damien@REDACTED 
> <mailto:damien@REDACTED>> wrote:
>
>     I'm sure this code is horrible from the point of view of a seasoned
>     Erlang developer, and I welcome any feedback. Especially regarding
>     speed
>     and robustness against faults.
>
>
> The key two questions to think about:
>
> * What happens if your spawned children die?
> * If a child locks up, does you system exhibit liveness/progress?
>
> Since you spawn children with spawn_link, they all link to the parent 
> process. So if one child dies, so does the parent. This gives you 
> all-or-nothing semantics: either all keys are returned, or the system 
> fails. An alternative semantics, based on spawn_monitor for instance, 
> could have you handle each key individually and return an error for 
> keys which fail.
First of all, thank you for spending time reading the code, and 
providing feedback !

The all-or-nothing semantic suits me : if something wrong happens while 
fetching any key, it's fine that the whole ReST request fails. The 
client should cope with that and retry, or whatever. However, I'll try 
to provide a better error, so that the client can know which key is at 
fault. Do I have to have a monitor semantic for that?

>
> Your receive expression never times out. So if a key fetch blocks, so 
> does your call. In some situations, when you can guarantee children 
> have progress, this is no problem, but it is often clever to add some 
> kind of timeout. However, when doing so, you must handle the case 
> where a blob arrives late to the process. This usually means you need 
> to make sure the mailbox flushes appropriately (which is something you 
> need to peruse the webmachine documentation for).
>
> It is often impossible to a-priori declare what kind of semantics you 
> want for these things. It is a weakness of "frameworks" that tries to 
> automate these kinds of things. You get a predetermined semantics, but 
> then you can't switch it up when you need a different semantics. So 
> you have to make the analysis of how you want your system to behave if 
> it starts failing.

Hm, so first of all yes I should have used a receive + timeout, I'll do 
that. However, your comment about making sure that the mailbox flushes 
appropriately worries me. Is the concern that the mailbox will become 
too big ? ( I don't think it's an issue as it's only RAM limited, right 
?) or is the concern the fact that webmachine could block some of the 
(late) signal to reach my process ? And I don't think I know where to 
begin to make the code robust against late replies. I'll go read 
webmachine doc though, but if you have any hint I'd be grateful.

>
> Finally, binary() values form a monoid under concatenation, with <<>> 
> as the neutral element. Thus you can write your binary join as
>
> binary_join(Parts) ->
>     lists:foldl(fun(P, Acc) -> <<Acc/binary, P/binary>> end, <<>>, Parts).
>
> The special case [] then returns <<>> and the case [X] forms << 
> (<<>>)/binary, X/binary>> which evaluates to X as expected. With these 
> changes, binary_join/1 is so simple it is probably worth just folding 
> into the call site directly.

wow thanks ! I was wondering why I had to write a binary_join myself and 
it wasn't provided in some standard library. That's because I don't need 
to write it :)

Thanks,
dams