Managing Experiments

Jeremy Raymond <>
Sun Jun 27 05:58:43 CEST 2010

I have a web application and need to manage functional 'experiments'. An
experiment is some functionality that's new that we want test out with a
subset of the application's users before rolling it out to everyone,
basically A/B testing new features. I'm looking for a clean way to manage
the code for experiments, in essence something equivalent to:

case user_in_experiment(UserId, new_feature_x) of
   true ->
      % do experiment functionality for new feature 'x'
   false ->
      % do current functionality

And then once we're happy with new feature 'x' make it the default
functionality. There may be several different experiments in the code at any
given time. Possible ways of accomplishing this that I've thought of are:

- case statements littered throughout the code
- encapsulating the functionality in funs and somehow do a lookup and select
the right one for the current user
- encapsulating the functionality in custom behaviors and somehow do a
lookup selecting the right one for the current user

Basically I need to be able to encapsulate and dynamically swap out
different bits of functionality throughout the application based upon who is
the current user. Anyone done something similar to this or have any insights
into how they might approach such a problem?

Jeremy Raymond

More information about the erlang-questions mailing list