[erlang-questions] Service platform (longish rant)

[Vlad Dumitrescu]

Hi,

On 5/8/07, Roger Larsson <roger.larsson@REDACTED> wrote:
> On Monday 07 May 2007 22:32, Vlad Dumitrescu wrote:
> > The example was
> >   </snip>  ...................
>
> But it can be simplified to
> </snip>

That was just an example, taken from a previous discussion.

> Note that all versions but your process based lacks one component from the
> Unix pipes - concurrency! The 'ls' is not required to finish before the
> final 'head', rather when head is finished the pipe is terminated.

Yes, that could be a possible requirement. However, there's nothing to
say that the compiler won't implement all links in the chain (even
functional ones) as simple processes.

> >     ls() => grep(_£_, "*.erl")   %% _£_ is a reserved variable,
> >                                          %%     meaning "the current
> > intermediate value"
> >           => sort                     %% this is a process name, could be a
> > pid => hd/1
>
> But what is the output from ls()? Complete listing? One line? One char?
>   ls() => sort => hd/1
> Same issue for sort, and hd/1... (sort would not like one char at the time...)

That is up to the particular service. Of course, services having
different format assumptions won't be easy to pipe together. But I
would personally choose a structured data format over the raw text
used in the Unix world.

For this particular example, I would have ls() return a list of file
descriptions if implemented as a function. The result list could be
lazily constructed behind the scenes, if the framework could handle it
transparently (by hiding the continuation passing infrastructure).

If ls() is implemented as a process, the result could be returned as a
sequence of messages containing list chunks, basically forming a lazy
list too.

[Yes, if ls() returns structured data, the next step shouldn't be
called 'grep', but again, it is an example]

> > If intermediate values need to be saved, writing
> >     Term => UnboundVariable => Next
> > would bind UnboundVariable to the current result value and
> > UnboundVariable will be accessible in the subsequent scope.
>
> This would require the complete result to be passed in each step...

Yes and no. There are two options:
* Some steps are eager, not lazy. The same applies to something like
sorting, where it's not useful to sort partial data.
* Or the value reflects the data sent until the current moment, and
even sort's output would be the sorted list of partial data. But this
is an even larger step away from Erlang as we know it, so it would
take a lot more thinking about what it really means. It feels like a
different computational model, data-flow based, where the '=>'
operator is defining a data stream between two entities. I have been
looking at that too.

I realize that these are delicate matters. I wanted to test the
response before going further.

best regards,
Vlad