1 Erl_Interface User's Guide

1.1  Introduction

The Erl_Interface library contains functions that help you integrate programs written in C and Erlang. The functions in Erl_Interface support the following:

• Manipulation of data represented as Erlang data types
• Conversion of data between C and Erlang formats
• Encoding and decoding of Erlang data types for transmission or storage
• Communication between C nodes and Erlang processes
• Backup and restore of C node state to and from Mnesia

Scope

In the following sections, these topics are described:

• Compiling your code for use with Erl_Interface
• Initializing Erl_Interface
• Encoding, decoding, and sending Erlang terms
• Building terms and patterns
• Pattern matching
• Connecting to a distributed Erlang node
• Using the Erlang Port Mapper Daemon (EPMD)
• Sending and receiving Erlang messages
• Remote procedure calls
• Using global names
• Using the registry

Prerequisites

It is assumed that the reader is familiar with the Erlang programming language.

1.2  Compiling and Linking Your Code

To use any of the Erl_Interface functions, include the following line in your code:

#include "ei.h"    

Determine where the top directory of your OTP installation is. To find this, start Erlang and enter the following command at the Eshell prompt:

Eshell V4.7.4  (abort with ^G)
1> code:root_dir().
/usr/local/otp    

To compile your code, ensure that your C compiler knows where to find ei.h by specifying an appropriate -I argument on the command line, or add it to the CFLAGS definition in your Makefile. The correct value for this path is $OTPROOT/lib/erl_interface-$EIVSN/include, where:

• $OTPROOT is the path reported by code:root_dir/0 in the example above.

• $EIVSN is the version of the Erl_Interface application, for example, erl_interface-3.2.3.

Compiling the code:

$ cc -c -I/usr/local/otp/lib/erl_interface-3.2.3/include myprog.c    

When linking:

• Specify the path to libei.a with -L$OTPROOT/lib/erl_interface-3.2.3/lib.
• Specify the name of the library with -lei.

Do this on the command line or add the flags to the LDFLAGS definition in your Makefile.

Linking the code:

$ ld -L/usr/local/otp/lib/erl_interface-3.2.3/
                            lib myprog.o -lei -o myprog    

On some systems it can be necessary to link with some more libraries (for example, libnsl.a and libsocket.a on Solaris, or wsock32.lib on Windows) to use the communication facilities of Erl_Interface.

If you use the Erl_Interface functions in a threaded application based on POSIX threads or Solaris threads, then Erl_Interface needs access to some of the synchronization facilities in your threads package. You must specify extra compiler flags to indicate which of the packages you use. Define _REENTRANT and either STHREADS or PTHREADS. The default is to use POSIX threads if _REENTRANT is specified.

1.3  Initializing the Library

Before calling any of the other functions in the library, initialize it by calling ei_init() exactly once.

1.4  Encoding, Decoding, and Sending Erlang Terms

Data sent between distributed Erlang nodes is encoded in the Erlang external format. You must therefore encode and decode Erlang terms into byte streams if you want to use the distribution protocol to communicate between a C program and Erlang.

The Erl_Interface library supports this activity. It has several C functions that create and manipulate Erlang data structures. The following example shows how to create and encode an Erlang tuple {tobbe,3928}:

ei_x_buff buf;
ei_x_new(&buf);
int i = 0;
ei_x_encode_tuple_header(&buf, 2);
ei_x_encode_atom(&buf, "tobbe");
ei_x_encode_long(&buf, 3928);    

For a complete description, see the ei module.

1.5  Building Terms

The previous example can be simplified by using the ei_x_format_wo_ver function to create an Erlang term:

ei_x_buff buf;
ei_x_new(&buf);
ei_x_format_wo_ver(&buf, "{~a,~i}", "tobbe", 3928);    

For a complete description of the different format directives, see the the ei_x_format_wo_ver function.

The following example is more complex:

ei_x_buff buf;
int i = 0;
ei_x_new(&buf);
ei_x_format_wo_ver(&buf,
                   "[{name,~a},{age,~i},{data,[{adr,~s,~i}]}]",
                   "madonna",
                   21,
                  "E-street", 42);
ei_print_term(stdout, buf.buff, &i);
ei_x_free(&buf);      

As in the previous examples, it is your responsibility to free the memory allocated for Erlang terms. In this example, ei_x_free() ensures that the data pointed to by buf is released.

1.6  Connecting to a Distributed Erlang Node

To connect to a distributed Erlang node, you must first initialize the connection routine with one of the ei_connect_init_* functions, which stores information, such as the hostname, and node name for later use:

int identification_number = 99;
int creation=1;
char *cookie="a secret cookie string"; /* An example */
const char* node_name = "einode@durin";
const char *cookie = NULL;
short creation = time(NULL) + 1;
ei_cnode ec;
ei_connect_init(ec,
                node_name,
                cookie,
                creation);    

For more information, see the ei_connect module.

After initialization, you set up the connection to the Erlang node. To specify the Erlang node you want to connect to, use the ei_connect_*() family of functions. The following example sets up the connection and is to result in a valid socket file descriptor:

int sockfd;
const char* node_name = "einode@durin"; /* An example */
if ((sockfd = ei_connect(ec, nodename)) < 0)
  fprintf(stderr, "ERROR: ei_connect failed");    

1.7  Using EPMD

erts:epmd is the Erlang Port Mapper Daemon. Distributed Erlang nodes register with epmd on the local host to indicate to other nodes that they exist and can accept connections. epmd maintains a register of node and port number information, and when a node wishes to connect to another node, it first contacts epmd to find the correct port number to connect to.

When you use ei_connect to connect to an Erlang node, a connection is first made to epmd and, if the node is known, a connection is then made to the Erlang node.

C nodes can also register themselves with epmd if they want other nodes in the system to be able to find and connect to them.

Before registering with epmd, you must first create a listen socket and bind it to a port. Then:

int pub;

pub = ei_publish(ec, port);    

pub is a file descriptor now connected to epmd. epmd monitors the other end of the connection. If it detects that the connection has been closed, the node becomes unregistered. So, if you explicitly close the descriptor or if your node fails, it becomes unregistered from epmd.

Notice that on some systems (such as VxWorks), a failed node is not detected by this mechanism, as the operating system does not automatically close descriptors that were left open when the node failed. If a node has failed in this way, epmd prevents you from registering a new node with the old name, as it thinks that the old name is still in use. In this case, you must close the port explicitly

1.8  Sending and Receiving Erlang Messages

Use one of the following two functions to send messages:

• ei_send
• ei_reg_send

As in Erlang, messages can be sent to a pid or to a registered name. It is easier to send a message to a registered name, as it avoids the problem of finding a suitable pid.

Use one of the following two functions to receive messages:

• ei_receive
• ei_receive_msg

Example of Sending Messages

In the following example, {Pid, hello_world} is sent to a registered process my_server:

ei_x_buff buf;
ei_x_new_with_version(&buf);

ei_x_encode_tuple_header(&buf, 2);
ei_x_encode_pid(&buf, ei_self(ec));
ei_x_encode_atom(&buf, "Hello world");

ei_reg_send(ec,fd,"my_server",buf,buf.index);

The first element of the tuple that is sent is your own pid. This enables my_server to reply. For more information about the primitives, see the ei_connect module.

Example of Receiving Messages

In this example, {Pid, Something} is received.

erlang_msg msg;
int index = 0;
int version;
int arity = 0;
erlang_pid pid;
ei_x_buff buf;
ei_x_new(&buf);
for (;;) {
  int got = ei_xreceive_msg(fd, &msg, &x);
  if (got == ERL_TICK)
    continue;
  if (got == ERL_ERROR) {
    fprintf(stderr, "ei_xreceive_msg, got==%d", got);
    exit(1);
  }
  break;
}
ei_decode_version(buf.buff, &index, &version);
ei_decode_tuple_header(buf.buff, &index, &arity);
if (arity != 2) {
  fprintf(stderr, "got wrong message");
  exit(1);
}
ei_decode_pid(buf.buff, &index, &pid);     

To provide robustness, a distributed Erlang node occasionally polls all its connected neighbors in an attempt to detect failed nodes or communication links. A node that receives such a message is expected to respond immediately with an ERL_TICK message. This is done automatically by ei_xreceive_msg(). However, when this has occurred, ei_xreceive_msg returns ERL_TICK to the caller without storing a message into the erlang_msg structure.

When a message has been received, it is the caller's responsibility to free the received message.

For more information, see the ei_connect and ei modules.

1.9  Remote Procedure Calls

An Erlang node acting as a client to another Erlang node typically sends a request and waits for a reply. Such a request is included in a function call at a remote node and is called a remote procedure call.

The following example checks if a specific Erlang process is alive:

int index = 0, is_alive;
ei_x_buff args, result;

ei_x_new(&result);
ei_x_new(&args);
ei_x_encode_list_header(&args, 1);
ei_x_encode_pid(&args, &check_pid);
ei_x_encode_empty_list(&args);

if (ei_rpc(&ec, fd, "erlang", "is_process_alive",
           args.buff, args.index, &result) < 0)
    handle_error();

if (ei_decode_version(result.buff, &index) < 0
    || ei_decode_bool(result.buff, &index, &is_alive) < 0)
    handle_error();    

For more information about ei_rpc() and its companions ei_rpc_to() and ei_rpc_from(), see the ei_connect module.

1.10  Using Global Names

A C node has access to names registered through the global module in Kernel. Names can be looked up, allowing the C node to send messages to named Erlang services. C nodes can also register global names, allowing them to provide named services to Erlang processes or other C nodes.

Erl_Interface does not provide a native implementation of the global service. Instead it uses the global services provided by a "nearby" Erlang node. To use the services described in this section, it is necessary to first open a connection to an Erlang node.

To see what names there are:

char **names;
int count;
int i;

names = ei_global_names(ec,fd,&count);

if (names) 
  for (i=0; i<count; i++) 
    printf("%s\n",names[i]);

free(names);    

ei_global_names allocates and returns a buffer containing all the names known to the global module in Kernel. count is initialized to indicate the number of names in the array. The array of strings in names is terminated by a NULL pointer, so it is not necessary to use count to determine when the last name is reached.

It is the caller's responsibility to free the array. ei_global_names allocates the array and all the strings using a single call to malloc(), so free(names) is all that is necessary.

To look up one of the names:

ETERM *pid;
char node[256];
erlang_pid the_pid;

if (ei_global_whereis(ec,fd,"schedule",&the_pid,node) < 0)
   fprintf(stderr, "ei_global_whereis error\n");    

If "schedule" is known to the global module in Kernel, an Erlang pid is written to the_pid. This pid that can be used to send messages to the schedule service. Also, node is initialized to contain the name of the node where the service is registered, so that you can make a connection to it by simply passing the variable to ei_connect.

Before registering a name, you should already have registered your port number with epmd. This is not strictly necessary, but if you neglect to do so, then other nodes wishing to communicate with your service cannot find or connect to your process.

Create a name that Erlang processes can use to communicate with your service:

ei_global_register(fd,servicename,ei_self(ec));    

After registering the name, use ei_accept to wait for incoming connections.

To unregister a name:

ei_global_unregister(ec,fd,servicename);    

1.11  Using the Registry

This section describes the use of the registry, a simple mechanism for storing key-value pairs in a C-node, as well as backing them up or restoring them from an Mnesia table on an Erlang node. For more detailed information about the individual API functions, see the registry module.

Keys are strings, that is, NULL-terminated arrays of characters, and values are arbitrary objects. Although integers and floating point numbers are treated specially by the registry, you can store strings or binary objects of any type as pointers.

To start, open a registry:

ei_reg *reg;

reg = ei_reg_open(45);    

The number 45 in the example indicates the approximate number of objects that you expect to store in the registry. Internally the registry uses hash tables with collision chaining, so there is no absolute upper limit on the number of objects that the registry can contain, but if performance or memory usage is important, then you are to choose a number accordingly. The registry can be resized later.

You can open as many registries as you like (if memory permits).

Objects are stored and retrieved through set and get functions. The following example shows how to store integers, floats, strings, and arbitrary binary objects:

struct bonk *b = malloc(sizeof(*b));
char *name = malloc(7);

ei_reg_setival(reg,"age",29); 
ei_reg_setfval(reg,"height",1.85);

strcpy(name,"Martin");
ei_reg_setsval(reg,"name",name); 

b->l = 42;
b->m = 12;
ei_reg_setpval(reg,"jox",b,sizeof(*b));    

If you try to store an object in the registry and there is an existing object with the same key, the new value replaces the old one. This is done regardless of whether the new object and the old one have the same type, so you can, for example, replace a string with an integer. If the existing value is a string or binary, it is freed before the new value is assigned.

Stored values are retrieved from the registry as follows:

long i;
double f;
char *s;
struct bonk *b;
int size;

i = ei_reg_getival(reg,"age");
f = ei_reg_getfval(reg,"height");
s = ei_reg_getsval(reg,"name");
b = ei_reg_getpval(reg,"jox",&size);    

In all the above examples, the object must exist and it must be of the right type for the specified operation. If you do not know the type of an object, you can ask:

struct ei_reg_stat buf;

ei_reg_stat(reg,"name",&buf);    

Buf is initialized to contain object attributes.

Objects can be removed from the registry:

ei_reg_delete(reg,"name");    

When you are finished with a registry, close it to remove all the objects and free the memory back to the system:

ei_reg_close(reg);    

Backing Up the Registry to Mnesia

The contents of a registry can be backed up to Mnesia on a "nearby" Erlang node. You must provide an open connection to the Erlang node (see ei_connect). Also, Mnesia 3.0 or later must be running on the Erlang node before the backup is initiated:

ei_reg_dump(fd, reg, "mtab", dumpflags);      

This example back up the contents of the registry to the specified Mnesia table "mtab". Once a registry has been backed up to Mnesia like this, more backups only affect objects that have been modified since the most recent backup, that is, objects that have been created, changed, or deleted. The backup operation is done as a single atomic transaction, so that either the entire backup is performed or none of it.

Likewise, a registry can be restored from a Mnesia table:

ei_reg_restore(fd, reg, "mtab");      

This reads the entire contents of "mtab" into the specified registry. After the restore, all the objects in the registry are marked as unmodified, so a later backup only affects objects that you have modified since the restore.

Notice that if you restore to a non-empty registry, objects in the table overwrite objects in the registry with the same keys. Also, the entire contents of the registry is marked as unmodified after the restore, including any modified objects that were not overwritten by the restore operation. This may not be your intention.

Storing Strings and Binaries

When string or binary objects are stored in the registry it is important that some simple guidelines are followed.

Most importantly, the object must have been created with a single call to malloc() (or similar), so that it can later be removed by a single call to free(). Objects are freed by the registry when it is closed, or when you assign a new value to an object that previously contained a string or binary.

Notice that if you store binary objects that are context-dependent (for example, containing pointers or open file descriptors), they lose their meaning if they are backed up to a Mnesia table and later restored in a different context.

When you retrieve a stored string or binary value from the registry, the registry maintains a pointer to the object and you are passed a copy of that pointer. You should never free an object retrieved in this manner because when the registry later attempts to free it, a runtime error occurs that likely causes the C-node to crash.

You are free to modify the contents of an object retrieved this way. However, when you do so, the registry is not aware of your changes, possibly causing it to be missed the next time you make an Mnesia backup of the registry contents. This can be avoided if you mark the object as dirty after any such changes with ei_reg_markdirty, or pass appropriate flags to ei_reg_dump.