erl_ddll

The erl_ddll module provides an interface for loading and unloading erlang linked in drivers in runtime.

The driver should be provided as a dynamically linked library in a object code format specific for the platform in use, i. e. .so files on most Unix systems and .ddl files on windows. An erlang linked in driver has to provide specific interfaces to the emulator, so this module is not designed for loading arbitrary dynamic libraries. For further information about erlang drivers, refer to the ERTS reference manual section erl_driver.

When describing a set of functions, (i.e. a module, a part of a module or an application) executing in a process and wanting to use a ddll-driver, we use the term user. There can be several users in one process (different modules needing the same driver) and several processes running the same code, making up several users of a driver. In the basic scenario, each user loads the driver before starting to use it and unloads the driver when done. The reference counting keeps track of processes as well as the number of loads by each process, so that the driver will only be unloaded when no one wants it (it has no user). The driver also keeps track of ports that are opened towards it, so that one can delay unloading until all ports are closed or kill all ports using the driver when it is unloaded.

The interface supports two basic scenarios of loading and unloading. Each scenario can also have the option of either killing ports when the driver is unloading, or waiting for the ports to close themselves. The scenarios are:

Load and unload on a "when needed basis"

This (most common) scenario simply supports that each user of the driver loads it when it is needed and unloads it when the user no longer have any use for it. The driver is always reference counted and as long as a process keeping the driver loaded is still alive, the driver is present in the system.

Each user of the driver use literally the same pathname for the driver when demanding load, but the users are not really concerned with if the driver is already loaded from the filesystem or if the object code has to be loaded from filesystem.

Two pairs of functions support this scenario:

load/2 and unload/1

When using the load/unload interfaces, the driver will not actually get unloaded until the last port using the driver is closed. The function unload/1 can return immediately, as the users are not really concerned with when the actual unloading occurs. The driver will actually get unloaded when no one needs it any longer.

If a process having the driver loaded dies, it will have the same effect as if unloading was done.

When loading, the function load/2 returns ok as soon as there is any instance of the driver present, so that if a driver is waiting to get unloaded (due to open ports), it will simply change state to no longer need unloading.

load_driver/2 and unload_driver/1

These interfaces is intended to be used when it is considered an error that ports are open towards a driver that no user has loaded. The ports still open when the last user calls unload_driver/1 or when the last process having the driver loaded dies, will get killed with reason driver_unloaded.

The function names load_driver and unload_driver are kept for backward compatibility.

Loading and reloading for code replacement

This scenario occurs when the driver code might need replacement during operation of the Erlang emulator. Implementing driver code replacement is somewhat more tedious than beam code replacement, as one driver cannot be loaded as both "old" and "new" code. All users of a driver must have it closed (no open ports) before the old code can be unloaded and the new code can be loaded.

The actual unloading/loading is done as one atomic operation, blocking all processes in the system from using the driver concerned while in progress.

The preferred way to do driver code replacement is to let one single process keep track of the driver. When the process start, the driver is loaded. When replacement is required, the driver is reloaded. Unload is probably never done, or done when the process exits. If more than one user has a driver loaded when code replacement is demanded, the replacement cannot occur until the last "other" user has unloaded the driver.

Demanding reload when a reload is already in progress is always an error. Using the high level functions, it is also an error to demand reloading when more than one user has the driver loaded. To simplify driver replacement, avoid designing your system so that more than than one user has the driver loaded.

The two functions for reloading drivers should be used together with corresponding load functions, to support the two different behaviors concerning open ports:

load/2 and reload/2

This pair of functions is used when reloading should be done after the last open port towards the driver is closed.

As reload/2 actually waits for the reloading to occur, a misbehaving process keeping open ports towards the driver (or keeping the driver loaded) might cause infinite waiting for reload. Timeouts has to be provided outside of the process demanding the reload or by using the low-level interface try_load/3 in combination with driver monitors (see below).

load_driver/2 and reload_driver/2

This pair of functions are used when open ports towards the driver should be killed with reason driver_unloaded to allow for new driver code to get loaded.

If, however, another process has the driver loaded, calling reload_driver returns the error code pending_process. As stated earlier, the recommended design is to not allow other users than the "driver reloader" to actually demand loading of the concerned driver.

Types:

Removes a driver monitor in much the same way as erlang:demonitor/1 does with process monitors. See monitor/2, try_load/3 and try_unload/2 for details about how to create driver monitors.

The function throws a badarg exception if the parameter is not a reference().

Types:

Returns a list of tuples {DriverName, InfoList}, where InfoList is the result of calling info/1 for that DriverName. Only dynamically linked in drivers are included in the list.

Types:

Returns a list of tuples {Tag, Value}, where Tag is the information item and Value is the result of calling info/2 with this driver name and this tag. The result being a tuple list containing all information available about a driver.

The different tags that will appear in the list are:

• processes
• driver_options
• port_count
• linked_in_driver
• permanent
• awaiting_load
• awaiting_unload

For a detailed description of each value, please read the description of info/2 below.

The function throws a badarg exception if the driver is not present in the system.

Types:

This function returns specific information about one aspect of a driver. The Tag parameter specifies which aspect to get information about. The Value return differs between different tags:

processes

Return all processes containing users of the specific drivers as a list of tuples {pid(),int()}, where the int() denotes the number of users in the process pid().

driver_options

Return a list of the driver options provided when loading, as well as any options set by the driver itself during initialization. The currently only valid option being kill_ports.

port_count

Return the number of ports (an int()) using the driver.

linked_in_driver

Return a bool(), being true if the driver is a statically linked in one and false otherwise.

permanent

Return a bool(), being true if the driver has made itself permanent (and is not a statically linked in driver). false otherwise.

awaiting_load

Return a list of all processes having monitors for loading active, each process returned as {pid(),int()}, where the int() is the number of monitors held by the process pid().

awaiting_unload

Return a list of all processes having monitors for unloading active, each process returned as {pid(),int()}, where the int() is the number of monitors held by the process pid().

If the options linked_in_driver or permanent return true, all other options will return the value linked_in_driver or permanent respectively.

The function throws a badarg exception if the driver is not present in the system or the tag is not supported.

Types:

Loads and links the dynamic driver Name. Path is a file path to the directory containing the driver. Name must be a sharable object/dynamic library. Two drivers with different Path parameters cannot be loaded under the same name. The Name is a string or atom containing at least one character.

The Name given should correspond to the filename of the actual dynamically loadable object file residing in the directory given as Path, but without the extension (i.e. .so). The driver name provided in the driver initialization routine must correspond with the filename, in much the same way as erlang module names correspond to the names of the .beam files.

If the driver has been previously unloaded, but is still present due to open ports against it, a call to load/2 will stop the unloading and keep the driver (as long as the Path is the same) and ok is returned. If one actually wants the object code to be reloaded, one uses reload/2 or the low-level interface try_load/3 instead. Please refer to the description of different scenarios for loading/unloading in the introduction.

If more than one process tries to load an already loaded driver withe the same Path, or if the same process tries to load it several times, the function will return ok. The emulator will keep track of the load/2 calls, so that a corresponding number of unload/2 calls will have to be done from the same process before the driver will actually get unloaded. It is therefore safe for an application to load a driver that is shared between processes or applications when needed. It can safely be unloaded without causing trouble for other parts of the system.

It is not allowed to load several drivers with the same name but with different Path parameters.

On success, the function returns ok. On failure, the return value is {error,ErrorDesc}, where ErrorDesc is an opaque term to be translated into human readable form by the format_error/1 function.

For more control over the error handling, again use the try_load/3 interface instead.

The function throws a badarg exception if the parameters are not given as described above.

Types:

Works essentially as load/2, but will load the driver with other options. All ports that are using the driver will get killed with the reason driver_unloaded when the driver is to be unloaded.

The number of loads and unloads by different users influence the actual loading and unloading of a driver file. The port killing will therefore only happen when the last user unloads the driver, or the last process having loaded the driver exits.

This interface (or at least the name of the functions) is kept for backward compatibility. Using try_load/3 with {driver_options,[kill_ports]} in the option list will give the same effect regarding the port killing.

The function throws a badarg exception if the parameters are not given as described above.

Types:

This function creates a driver monitor and works in many ways as the function erlang:monitor/2, does for processes. When a driver changes state, the monitor results in a monitor-message being sent to the calling process. The MonitorRef returned by this function is included in the message sent.

As with process monitors, each driver monitor set will only generate one single message. The monitor is "destroyed" after the message is sent and there is then no need to call demonitor/1.

The MonitorRef can also be used in subsequent calls to demonitor/1 to remove a monitor.

The function accepts the following parameters:

Tag

The monitor tag is always driver as this function can only be used to create driver monitors. In the future, driver monitors will be integrated with process monitors, why this parameter has to be given for consistence.

Item

The Item parameter specifies which driver one wants to monitor (the name of the driver) as well as which state change one wants to monitor. The parameter is a tuple of arity two whose first element is the driver name and second element is either of:

loaded

Notify me when the driver is reloaded (or loaded if loading is underway). It only makes sense to monitor drivers that are in the process of being loaded or reloaded. One cannot monitor a future-to-be driver name for loading, that will only result in a 'DOWN' message being immediately sent. Monitoring for loading is therefore most useful when triggered by the try_load/3 function, where the monitor is created because the driver is in such a pending state.

Setting a driver monitor for loading will eventually lead to one of the following messages being sent:

{'UP', reference(), driver, Name, loaded}

This message is sent, either immediately if the driver is already loaded and no reloading is pending, or when reloading is executed if reloading is pending.

The user is expected to know if reloading is demanded prior to creating a monitor for loading.

{'UP', reference(), driver, Name, permanent}

This message will be sent if reloading was expected, but the (old) driver made itself permanent prior to reloading. It will also be sent if the driver was permanent or statically linked in when trying to create the monitor.

{'DOWN', reference(), driver, Name, load_cancelled}

This message will arrive if reloading was underway, but the user having requested reload cancelled it by either dying or calling try_unload/2 (or unload/1/unload_driver/1) again before it was reloaded.

{'DOWN', reference(), driver, Name, {load_failure, Failure}}

This message will arrive if reloading was underway but the loading for some reason failed. The Failure term is one of the errors that can be returned from try_load/3. The error term can be passed to format_error/1 for translation into human readable form. Note that the translation has to be done in the same running erlang virtual machine as the error was detected in.

unloaded

Monitor when a driver gets unloaded. If one monitors a driver that is not present in the system, one will immediately get notified that the driver got unloaded. There is no guarantee that the driver was actually ever loaded.

A driver monitor for unload will eventually result in one of the following messages being sent:

{'DOWN', reference(), driver, Name, unloaded}

The driver instance monitored is now unloaded. As the unload might have been due to a reload/2 request, the driver might once again have been loaded when this message arrives.

{'UP', reference(), driver, Name, unload_cancelled}

This message will be sent if unloading was expected, but while the driver was waiting for all ports to get closed, a new user of the driver appeared and the unloading was cancelled.

This message appears when an {ok, pending_driver}) was returned from try_unload/2) for the last user of the driver and then a {ok, already_loaded} is returned from a call to try_load/3.

If one wants to really monitor when the driver gets unloaded, this message will distort the picture, no unloading was really done. The unloaded_only option creates a monitor similar to an unloaded monitor, but does never result in this message.

{'UP', reference(), driver, Name, permanent}

This message will be sent if unloading was expected, but the driver made itself permanent prior to unloading. It will also be sent if trying to monitor a permanent or statically linked in driver.

unloaded_only

A monitor created as unloaded_only behaves exactly as one created as unloaded with the exception that the {'UP', reference(), driver, Name, unload_cancelled} message will never be sent, but the monitor instead persists until the driver really gets unloaded.

The function throws a badarg exception if the parameters are not given as described above.

Types:

Reloads the driver named Name from a possibly different Path than was previously used. This function is used in the code change scenario described in the introduction.

If there are other users of this driver, the function will return {error, pending_process}, but if there are no more users, the function call will hang until all open ports are closed.

If one wants to avoid hanging on open ports, one should use the try_load/3 function instead.

The Name and Path parameters have exactly the same meaning as when calling the plain load/2 function.

On success, the function returns ok. On failure, the function returns an opaque error, with the exception of the pending_process error described above. The opaque errors are to be translated into human readable form by the format_error/1 function.

For more control over the error handling, again use the try_load/3 interface instead.

The function throws a badarg exception if the parameters are not given as described above.

Types:

Works exactly as reload/2, but for drivers loaded with the load_driver/2 interface.

As this interface implies that ports are being killed when the last user disappears, the function wont hang waiting for ports to get closed.

For further details, see the scenarios in the module description and refer to the reload/2 function description.

The function throws a badarg exception if the parameters are not given as described above.

Types:

This function provides more control than the load/2/reload/2 and load_driver/2/reload_driver/2 interfaces. It will never wait for completion of other operations related to the driver, but immediately return the status of the driver as either:

{ok, loaded}

The driver was actually loaded and is immediately usable.

{ok, already_loaded}

The driver was already loaded by another process and/or is in use by a living port. The load by you is registered and a corresponding try_unload is expected sometime in the future.

{ok, pending_driver}or {ok, pending_driver, reference()}

The load request is registered, but the loading is delayed due to the fact that an earlier instance of the driver is still waiting to get unloaded (there are open ports using it). Still, unload is expected when you are done with the driver. This return value will mostly happen when the {reload,pending_driver} or {reload,pending} options are used, but can happen when another user is unloading a driver in parallel and the kill_ports driver option is set. In other words, this return value will always need to be handled!

{ok, pending_process}or {ok, pending_process, reference()}

The load request is registered, but the loading is delayed due to the fact that an earlier instance of the driver is still waiting to get unloaded by another user (not only by a port, in which case {ok,pending_driver} would have been returned). Still, unload is expected when you are done with the driver. This return value will only happen when the {reload,pending} option is used.

When the function returns {ok, pending_driver} or {ok, pending_process}, one might want to get information about when the driver is actually loaded. This can be achieved by using the {monitor, PendingOption} option.

When monitoring is requested, and a corresponding {ok, pending_driver} or {ok, pending_process} would be returned, the function will instead return a tuple {ok, PendingStatus, reference()} and the process will, at a later time when the driver actually gets loaded, get a monitor message. The monitor message one can expect is described in the monitor/2 function description.

The function accepts the following parameters:

Path

The filesystem path to the directory where the driver object file is situated. The filename of the object file (minus extension) must correspond to the driver name (used in the name parameter) and the driver must identify itself with the very same name. The Path might be provided as an io_list, meaning it can be a list of other io_lists, characters (eight bit integers) or binaries, all to be flattened into a sequence of characters.

The (possibly flattened) Path parameter must be consistent throughout the system, a driver should, by all users, be loaded using the same literalPath. The exception is when reloading is requested, in which case the Path may be specified differently. Note that all users trying to load the driver at a later time will need to use the newPath if the Path is changed using a reload option. This is yet another reason to have only one loader of a driver one wants to upgrade in a running system!

Name

The name parameter is the name of the driver to be used in subsequent calls to open_port. The name can be specified either as an io_list() or as an atom(). The name given when loading is used to find the actual object file (with the help of the Path and the system implied extension suffix, i.e. .so). The name by which the driver identifies itself must also be consistent with this Name parameter, much as a beam-file's module name much correspond to its filename.

OptionList

A number of options can be specified to control the loading operation. The options are given as a list of two-tuples, the tuples having the following values and meanings:

{driver_options, DriverOptionsList}

This option is to provide options that will change its general behavior and will "stick" to the driver throughout its lifespan.

The driver options for a given driver name need always to be consistent, even when the driver is reloaded, meaning that they are as much a part of the driver as the actual name.

Currently the only allowed driver option is kill_ports, which means that all ports opened towards the driver are killed with the exit-reason driver_unloaded when no process any longer has the driver loaded. This situation arises either when the last user calls try_unload/2, or the last process having loaded the driver exits.

{monitor, MonitorOption}

A MonitorOption tells try_load/3 to trigger a driver monitor under certain conditions. When the monitor is triggered, the function will return a three-tuple {ok, PendingStatus, reference()}, where the reference() is the monitor ref for the driver monitor.

Only one MonitorOption can be specified and it is either the atom pending, which means that a monitor should be created whenever a load operation is delayed, and the atom pending_driver, in which a monitor is created whenever the operation is delayed due to open ports towards an otherwise unused driver. The pending_driver option is of little use, but is present for completeness, it is very well defined which reload-options might give rise to which delays. It might, however, be a good idea to use the same MonitorOption as the ReloadOption if present.

If reloading is not requested, it might still be useful to specify the monitor option, as forced unloads (kill_ports driver option or the kill_ports option to try_unload/2) will trigger a transient state where driver loading cannot be performed until all closing ports are actually closed. So, as try_unload can, in almost all situations, return {ok, pending_driver}, one should always specify at least {monitor, pending_driver} in production code (see the monitor discussion above).

{reload,RealoadOption}

This option is used when one wants to reload a driver from disk, most often in a code upgrade scenario. Having a reload option also implies that the Path parameter need not be consistent with earlier loads of the driver.

To reload a driver, the process needs to have previously loaded the driver, i.e there has to be an active user of the driver in the process.

The reload option can be either the atom pending, in which reloading is requested for any driver and will be effectuated when all ports opened against the driver are closed. The replacement of the driver will in this case take place regardless of if there are still pending users having the driver loaded! The option also triggers port-killing (if the kill_ports driver option is used) even though there are pending users, making it usable for forced driver replacement, but laying a lot of responsibility on the driver users. The pending option is seldom used as one does not want other users to have loaded the driver when code change is underway.

The more useful option is pending_driver, which means that reloading will be queued if the driver is not loaded by any other users, but the driver has opened ports, in which case {ok, pending_driver} will be returned (a monitor option is of course recommended).

If the driver is unloaded (not present in the system), the error code not_loaded will be returned. The reload option is intended for when the user has already loaded the driver in advance.

The function might return numerous errors, of which some only can be returned given a certain combination of options.

A number of errors are opaque and can only be interpreted by passing them to the format_error/1 function, but some can be interpreted directly:

{error,linked_in_driver}

The driver with the specified name is an erlang statically linked in driver, which cannot be manipulated with this API.

{error,inconsistent}

The driver has already been loaded with either other DriverOptions or a different literalPath argument.

This can happen even if a reload option is given, if the DriverOptions differ from the current.

{error, permanent}

The driver has requested itself to be permanent, making it behave like an erlang linked in driver and it can no longer be manipulated with this API.

{error, pending_process}

The driver is loaded by other users when the {reload, pending_driver} option was given.

{error, pending_reload}

Driver reload is already requested by another user when the {reload, ReloadOption} option was given.

{error, not_loaded_by_this_process}

Appears when the reload option is given. The driver Name is present in the system, but there is no user of it in this process.

{error, not_loaded}

Appears when the reload option is given. The driver Name is not in the system. Only drivers loaded by this process can be reloaded.

All other error codes are to be translated by the format_error/1 function. Note that calls to format_error should be performed from the same running instance of the erlang virtual machine as the error was detected in, due to system dependent behavior concerning error values.

If the arguments or options are malformed, the function will throw a badarg exception.

Types:

This is the low level function to unload (or decrement reference counts of) a driver. It can be used to force port killing, in much the same way as the driver option kill_ports implicitly does, and it can trigger a monitor either due to other users still having the driver loaded or that there are open ports using the driver.

Unloading can be described as the process of telling the emulator that this particular part of the code in this particular process (i.e. this user) no longer needs the driver. That can, if there are no other users, trigger actual unloading of the driver, in which case the driver name disappears from the system and (if possible) the memory occupied by the driver executable code is reclaimed. If the driver has the kill_ports option set, or if kill_ports was specified as an option to this function, all pending ports using this driver will get killed when unloading is done by the last user. If no port-killing is involved and there are open ports, the actual unloading is delayed until there are no more open ports using the driver. If, in this case, another user (or even this user) loads the driver again before the driver is actually unloaded, the unloading will never take place.

To allow the user that requests unloading to wait for actual unloading to take place, monitor triggers can be specified in much the same way as when loading. As users of this function however seldom are interested in more than decrementing the reference counts, monitoring is more seldom needed. If the kill_ports option is used however, monitor trigging is crucial, as the ports are not guaranteed to have been killed until the driver is unloaded, why a monitor should be triggered for at least the pending_driver case.

The possible monitor messages that can be expected are the same as when using the unloaded option to the monitor/2 function.

The function will return one of the following statuses upon success:

{ok, unloaded}

The driver was immediately unloaded, meaning that the driver name is now free to use by other drivers and, if the underlying OS permits it, the memory occupied by the driver object code is now reclaimed.

The driver can only be unloaded when there are no open ports using it and there are no more users requiring it to be loaded.

{ok, pending_driver}or {ok, pending_driver, reference()}

This return value indicates that this call removed the last user from the driver, but there are still open ports using it. When all ports are closed and no new users have arrived, the driver will actually be reloaded and the name and memory reclaimed.

This return value is valid even when the option kill_ports was used, as killing ports may not be a process that completes immediately. The condition is, in that case, however transient. Monitors are as always useful to detect when the driver is really unloaded.

{ok, pending_process}or {ok, pending_process, reference()}

The unload request is registered, but there are still other users holding the driver. Note that the term pending_process might refer to the running process, there might be more than one user in the same process.

This is a normal, healthy return value if the call was just placed to inform the emulator that you have no further use of the driver. It is actually the most common return value in the most common scenario described in the introduction.

The function accepts the following parameters:

Name

The name parameter is the name of the driver to be unloaded. The name can be specified either as an io_list() or as an atom().

OptionList

The OptionList argument can be used to specify certain behavior regarding ports as well as triggering monitors under certain conditions:

kill_ports

Force killing of all ports opened using this driver, with the exit reason driver_unloaded, if you are the lastuser of the driver.

If there are other users having the driver loaded, this option will have no effect.

If one wants the consistent behavior of killing ports when the last user unloads, one should use the driver option kill_ports when loading the driver instead.

{monitor, MonitorOption}

This option creates a driver monitor if the condition given in MonitorOptions is true. The valid options are:

pending_driver

Create a driver monitor if the return value is to be {ok, pending_driver}.

pending

Create a monitor if the return value will be either {ok, pending_driver} or {ok, pending_process}.

The pending_driver MonitorOption is by far the most useful and it has to be used to ensure that the driver has really been unloaded and the ports closed whenever the kill_ports option is used or the driver may have been loaded with the kill_ports driver option.

By using the monitor-triggers in the call to try_unload one can be sure that the monitor is actually added before the unloading is executed, meaning that the monitor will always get properly triggered, which would not be the case if one called erl_ddll:monitor/2 separately.

The function may return several error conditions, of which all are well specified (no opaque values):

{error, linked_in_driver}

You were trying to unload an erlang statically linked in driver, which cannot be manipulated with this interface (and cannot be unloaded at all).

{error, not_loaded}

The driver Name is not present in the system.

{error, not_loaded_by_this_process}

The driver Name is present in the system, but there is no user of it in this process.

As a special case, drivers can be unloaded from processes that has done no corresponding call to try_load/3 if, and only if, there are no users of the driver at all, which may happen if the process containing the last user dies.

{error, permanent}

The driver has made itself permanent, in which case it can no longer be manipulated by this interface (much like a statically linked in driver).

The function throws a badarg exception if the parameters are not given as described above.

Types:

Unloads, or at least dereferences the driver named Name. If the caller is the last user of the driver, and there are no more open ports using the driver, the driver will actually get unloaded. In all other cases, actual unloading will be delayed until all ports are closed and there are no remaining users.

If there are other users of the driver, the reference counts of the driver is merely decreased, so that the caller is no longer considered a user of the driver. For usage scenarios, see the description in the beginning of this document.

The ErrorDesc returned is an opaque value to be passed further on to the format_error/1 function. For more control over the operation, use the try_unload/2 interface.

The function throws a badarg exception if the parameters are not given as described above.

Types:

Unloads, or at least dereferences the driver named Name. If the caller is the last user of the driver, all remaining open ports using the driver will get killed with the reason driver_unloaded and the driver will eventually get unloaded.

If there are other users of the driver, the reference counts of the driver is merely decreased, so that the caller is no longer considered a user. For usage scenarios, see the description in the beginning of this document.

The ErrorDesc returned is an opaque value to be passed further on to the format_error/1 function. For more control over the operation, use the try_unload/2 interface.

The function throws a badarg exception if the parameters are not given as described above.

Types:

Returns a list of all the available drivers, both (statically) linked-in and dynamically loaded ones.

The driver names are returned as a list of strings rather than a list of atoms for historical reasons.

More information about drivers can be obtained using one of the info functions.

Types:

Takes an ErrorDesc returned by load, unload or reload functions and returns a string which describes the error or warning.

erl_driver(4), driver_entry(4)