[erlang-questions] ETS performance visualisation

[Kacper Mentel]

Hi!

My name is Kacper and I'm participating in this year's edition of Google
Summer of Code under BEAM Community. My GSoC's project is an extension to
Erlang Performance Lab. Erlang Lab is an Open Source project aimed at
analysing the performance of Erlang and Elixir systems. The project
provides web-based visualisations of the analysis. It can help developers
to better understand their systems, observe the system behaviour and
identify weaknesses.

My work is focused on visualising ETS tables performance. If you want to
learn more about my GSoC project or erlangpl here are the links to have a
look at:

https://summerofcode.withgoogle.com/projects/#5695385677856768
https://github.com/erlanglab/erlangpl
https://github.com/mkacper/erlangpl

The extension to the EPL I'm going to implement will deliver 3 views which
can be seen as layers through which you can look at the ETS tables. The
idea is simple — if you want to trace a problem related to an ETS table,
you start from the top-level view (the most general one) and go down
through the subsequent layers (views) until you reach the one that will
help you identify the source of the problem you are looking for. The scheme
of the views looks as follows: cluster view (shows all the nodes in a
cluster) -> node view (shows all ETS tables on a node) -> ETS details
(shows more detailed information).

I've implemented the first view so far. Here is the screenshot:
https://drive.google.com/file/d/0BzXT6yCa9ckQVzF
0RnhibG9LbFE/view?usp=sharing

Now I'm working on the second view (node view). Here are some screenshots:
https://drive.google.com/file/d/0BzXT6yCa9ckQTXNuTzYzVW00c1U
/view?usp=sharing
https://drive.google.com/file/d/0BzXT6yCa9ckQS05QQ3pWeWI5bkU
/view?usp=sharing

It shows all the ETS tables from a node linked to their owner's processes.
The color of the tables reflects the amount of allocated memory for the
table (red -> the heaviest, blue -> the lightest). This is only an example
how we can visualise ETS related data.

The most imprtant questions to you are:
- What is worth measuring and visualising in terms of ETS performance?
- What kind of information can help developers to find ETS related problems?

Here are some of my (and EPL team's) ideas:

- the amount of the allocated memory for a particular ETS table,
- memory utilization/fragmentation - real size of a particular table in
bytes compared to the amount of the memory allocated for that table
- number of rows in a partiuclar table
- some statistics about locks protecting ETS tables (collected using lcnt
module)
- mean acces time to the table in a unit of time

I would be very grateful for your feedback, suggestions and ideas.

Cheers,
Kacper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://erlang.org/pipermail/erlang-questions/attachments/20170705/7a2bd83f/attachment.htm>