[erlang-questions] A rendering server in Erlang? Performance gains?

[bryan rasmussen]

Hi,

I'm supposed to make a Rendering Server, defined as a Server that
takes a number of file formats as inputs and allows the output of a
number of other filetypes. For example if one inputs a PDF it might be
output as both TXT, TIFF, Word, Open Office, if one inputs a TIFF it
might be output as TXT and PDF.

Basically it should be possible to slot new format
conversion/rendering services in.
It does not need to be over HTTP but I want it to be, mainly because
if I can't get it okayed to be over HTTP it will come to have some
other non-http compatible functionality that will end up not making it
scale very well, as well as being prone to security hazards and
needing much more maintanence. So, like I said, I would prefer it to
work over HTTP.


SSL security needs to be enabled.

There are some specific requirements for technology as follows:

1. Must run on Windows because
2. Microsoft Office must be installed because
 - guy in charge with seniority says he wants MS Office installed.
(despite his also insisting that it should be a big piece of hardware
that sits in the server room and nobody ever has to do any maintenance
or fiddling with it because it is just so secure and if anything fails
it just keeps going and going and so forth....)

What I am thinking, initially, is the following:

a service endpoint takes the File to be converted (or files, the first
version doesn't need to support mime multipart messages but it will
need to in the future) via a POST.
The response code is a 303 See other.
The uri it would point to would be

http://{server}/outputs/{timestamp}/{filename}/render


A request to that would return a dynamically generated XML file
<RenderedResult>
...
</RenderedResult>
That would specify the status of the various rendering operations.
Thus if one should have a rendering of PDF to TIFF then

<RenderedResult>
<output type="TIFF" status="complete"/>
</RenderedResult>

Would tell the client that the TIFF file was complete and they could
request that file via
http://{server}/outputs/{timestamp}/{filename}/render/TIFF


I think that part of the application seems to be something that Erlang
(with YAWS - how easy is it to specify Custom HTTP headers and process
the same with YAWS, btw? ) will produce benefits for.

The problem is in the rendering.
The service takes the file to be rendered and writes it to a folder in
the file system.

This folder is monitored by the renderer dispatcher.

The renderer dispatcher takes the new files and sends them to the
appropriate renderer. This will probably need to be via the Command
Line (since we do not want to have to write Renderers for every file
rendering we support in the future, instead we want to use various
tools etc. However the aforementioned person in authority also wants
us to use some particular components to do things, specifically
http://www.peernet.com/epro/index.html and
http://buy.abbyy.com/content/frpro/default.aspx I'm not sure why these
particular components are wanted and not others but they're the ones
that are specified - as well as the specification of MS Windows and MS
Office running on the server). The programs that do the individual
rendering will not be  be written in erlang. Thus I suppose I do not
gain any benefit from erlang in that context.

The output from these processes is taken and put into the location
where the RenderedResult document can be generated.
Errors in the processes doing rendering must be logged.
It will also be an error if a process takes to long and such a process
should be shut down to free up system resources.
Thus if Erlang could run a Windows process, look for it to generate
errors and shut it down?


Can anyone see any point in the rendering process where Erlang could
give performance benefits? If so how should I maximally structure the
solution to take advantage of Erlang and to minimize non-erlang
process instabilities, performance bottlenecks etc. ?

I hope my phrasing has been relatively clear and non-problematic. I
have left details out that I figured was just extraneous functionality
that would not be especially affected by the decision to use erlang or
not - for example specific information as to the posts, responses,
redirects, http headers and such of the proposed web service.


Cheers,
Bryan Rasmussen