[erlang-questions] netmap, UDP, and Erlang

[Fred Hebert]

On 03/25, Richard O'Keefe wrote:
>I need to do some load testing between two machines;
>the protocol is an astronomy protocol layered over UDP
>and the data rate is higher than anything I've had my
>hands on before.  The system is going to be Ubuntu 18 LTS
>and while *eventually* there is going to be a custom FPGA
>between the line and the CPU it would be nice to use
>something other than Python or C++ for prototyping.

I've been on a team that has experimented with netmap on both FreeBSD 
and Linux (Ubuntu) for UDP traffic handling, although that code base was 
mostly maintained in C++ itself rather than Erlang (the latter was used 
to manage the C++ code, among other things).

>
>1. Does netmap live up to its promise?

Yes. It's pretty damn fast compared to all other alternatives we'd had 
looked into at the time, although there are some different alternatives 
on Linux that we have not tried.

>2. Are there there any issues installing it on
>   Ubuntu 18?

Mostly FreeBSD was really nice with it since it shipped with the kernel 
module required for netmap. Linux, by comparison, required the 
compilation of custom kernel modules, which became rather painful. Every 
time we wanted to upgrade the linux kernel, we also had to recompile all 
modules, which made for very complex upgrade sequences for devices in 
the field.

>3. Has anyone used this with Erlang/got any necessary
>   glue code?
>

Unfortunately not; all of it was private, and only talked to Erlang 
through more isolated mechanisms than NIFs or anything of the kind.

>If this is a bad idea,
>I'd find it helpful to be told why.
>"A little learning is a dangerous thing"
>and I am wary of the Dunning-Kruger effect.

The big problem is really the poor support for netmap to be built as a 
kernel module on Ubuntu. Otherwise it works.
Other things to look into might be those named at 
https://lwn.net/Articles/719850/ -- namely af_packet which I heard 
should be easier to deal with, but have no experience with.

Maybe things changed since then, but those were the last echoes I had 
heard about it all.