Skip to content

About

Elixir A/B testing SDK for cross-SDK tests

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

96 Commits

Folders and files

Repository files navigation

ABsmartly Elixir SDK

The official Elixir SDK for ABsmartly - A/B testing and feature flagging platform.

Compatibility

The ABsmartly Elixir SDK is compatible with Elixir 1.17+ and Erlang/OTP 25+. It provides both synchronous and asynchronous context creation for variant assignment and goal tracking. The Context is implemented as a GenServer process.

Installation

Add absmartly to your list of dependencies in mix.exs:

def deps do
  [
    {:absmartly, "~> 1.0"}
  ]
end

Then run:

mix deps.get

Getting Started

Please follow the installation instructions before trying the following code.

Initialization

This example assumes an API Key, an Application, and an Environment have been created in the ABsmartly web console.

alias ABSmartly.SDK

{:ok, sdk} = SDK.new(
  endpoint: "https://your-company.absmartly.io/v1",
  api_key: System.get_env("ABSMARTLY_API_KEY"),
  application: "website",
  environment: "production"
)

With Optional Parameters

{:ok, sdk} = SDK.new(
  endpoint: "https://your-company.absmartly.io/v1",
  api_key: System.get_env("ABSMARTLY_API_KEY"),
  application: "website",
  environment: "production",
  timeout: 5000,
  retries: 3
)

SDK Options

Config Type Required? Default Description
endpoint String ✅ nil The URL to your API endpoint. Most commonly "https://your-company.absmartly.io/v1"
api_key String ✅ nil Your API key which can be found on the Web Console.
environment String ✅ nil The environment of the platform where the SDK is installed. Environments are created on the Web Console and should match the available environments in your infrastructure.
application String ✅ nil The name of the application where the SDK is installed. Applications are created on the Web Console and should match the applications where your experiments will be running.
retries Integer ❌ 5 Number of retry attempts for failed HTTP requests
timeout Integer ❌ 3000 Connection timeout in milliseconds

Creating a New Context

Synchronously

alias ABSmartly.{SDK, Context}

units = %{"session_id" => "5ebf06d8cb5d8137290c4abb64155584fbdb64d8"}

{:ok, context} = SDK.create_context(sdk, units)

true = Context.is_ready?(context)

Asynchronously

alias ABSmartly.{SDK, Context}

units = %{"session_id" => "5ebf06d8cb5d8137290c4abb64155584fbdb64d8"}

{:ok, context} = SDK.create_context_async(sdk, units)

Context.wait_until_ready(context)

With Pre-fetched Data

When doing full-stack experimentation with ABsmartly, we recommend creating a context only once on the server-side. Creating a context involves a round-trip to the ABsmartly event collector. We can avoid repeating the round-trip on the client-side by sending the server-side data embedded in the first document.

alias ABSmartly.{SDK, Context, Types}

units = %{"session_id" => "5ebf06d8cb5d8137290c4abb64155584fbdb64d8"}

context_data = %Types.ContextData{
  experiments: [...]
}

{:ok, context} = SDK.create_context_with(sdk, units, context_data)

true = Context.is_ready?(context)

Refreshing the Context with Fresh Experiment Data

For long-running contexts, the context can be refreshed manually to pull updated experiment data.

new_data = %{"experiments" => [...]}

:ok = Context.refresh(context, new_data)

Setting Extra Units

You can add additional units to a context by calling the set_unit/3 function. This may be used, for example, when a user logs in to your application, and you want to use the new unit type in the context.

Note: You cannot override an already set unit type as that would be a change of identity. In this case, you must create a new context instead. The set_unit/3 and set_units/2 functions can be called before the context is ready.

Context.set_unit(context, "db_user_id", "1000013")

Context.set_units(context, %{
  "db_user_id" => "1000013",
  "user_type" => "premium"
})

Basic Usage

Selecting a Treatment

variant = Context.treatment(context, "exp_test_experiment")

case variant do
  0 ->
    # User is in control group (variant 0)
    render_control_experience()
  _ ->
    # User is in treatment group
    render_treatment_experience()
end

Treatment Variables

Variables allow you to change experiment parameters without code deployment.

button_color = Context.variable_value(context, "button_color", "blue")

timeout = Context.variable_value(context, "api_timeout", 5000)

Peek at Treatment Variants

Although generally not recommended, it is sometimes necessary to peek at a treatment without triggering an exposure. The ABsmartly SDK provides a peek/2 function for that.

variant = Context.peek(context, "exp_test_experiment")

if variant == 0 do
  # User is in control group
else
  # User is in treatment group
end

Peeking at Variables

color = Context.peek_variable_value(context, "button_color", "blue")

Overriding Treatment Variants

During development or testing, you may want to force a specific variant. The set_override/3 function allows you to override the variant assignment. The set_override/3 and set_overrides/2 functions can be called before the context is ready.

Context.set_override(context, "exp_test_experiment", 1)

Context.set_overrides(context, %{
  "exp_test_experiment" => 1,
  "exp_another_test" => 0
})

Advanced

Context Attributes

Attributes are used for audience targeting. The set_attribute/3 function can be called before the context is ready. It accepts native Elixir types directly.

Context.set_attribute(context, "user_agent", "Mozilla/5.0")
Context.set_attribute(context, "customer_age", "new_customer")
Context.set_attribute(context, "age", 25)

Context.set_attributes(context, %{
  "user_agent" => "Mozilla/5.0",
  "customer_age" => "new_customer",
  "age" => 25
})

Audience filters can also use application and environment, the application and environment names configured in the SDK. They are always available, but they are not real attributes: they are not published (the collector adds them on its side) and get_attributes/1 does not return them. Names that are not configured or empty are left out. An attribute you set with the same name, at any time, takes precedence:

Context.set_attribute(context, "environment", "staging") # audiences now see "staging"

Audience filters and assignment-rule conditions can compare versions with semver_eq, semver_gt, semver_gte, semver_lt and semver_lte. A leading v and any +build metadata are ignored, missing parts count as 0 (1.2 equals 1.2.0), numeric parts compare as numbers (1.10.0 is greater than 1.9.0) and a pre-release sorts before its release (1.0.0-beta is less than 1.0.0). A value that is missing or empty matches none of them.

{"filter": [{"semver_gte": [{"var": "app_version"}, {"value": "2.4.0"}]}]}

Assignment Rules

An experiment's assignment rules, set in the Web Console, are evaluated in order, and the first matching rule takes precedence over audience targeting, traffic allocation, the full-on variant and custom assignments. Only set_override/3 wins over a rule.

  • An assign rule gives a fixed variant.
  • A split rule randomizes the unit over its own percentages, e.g. 20/30/50. While the experiment's unit is missing, a split rule gives the full-on variant (0 for an experiment that is not full-on); once the unit is set, the next read randomizes it, unless that variant was already exposed.

Rule conditions see the same attributes as audience filters, including application and environment. Rules can also be scoped to environments; a rule scoped to environments never matches when the SDK has no environment configured. If the first matching rule's variant is out of range, or its split does not have one percentage per variant, normal assignment applies and later rules are not tried. Rules that are not valid are skipped.

Exposures from a rule are published with ruleOverride: true (and assigned: false) so they can be excluded from analysis. A cached assignment is re-evaluated, and exposed again, when an attribute change or a refresh changes the outcome of the rules.

Custom Assignments

Custom assignments allow you to implement your own assignment logic while still benefiting from ABsmartly's analytics.

Context.set_custom_assignment(context, "exp_test_experiment", 1)

Context.set_custom_assignments(context, %{
  "exp_test_experiment" => 1,
  "exp_another_experiment" => 0
})

Tracking Goals

Goals are created in the ABsmartly web console.

Context.track(context, "signup")

Context.track(context, "purchase", %{
  "amount" => 99.99,
  "items" => 3
})

Publishing Pending Data

Sometimes it is necessary to ensure all events have been published to the ABsmartly collector before proceeding. You can explicitly call the publish/1 function.

:ok = Context.publish(context)

If the publisher fails, publish/1 returns {:error, reason}, emits an error event and keeps the pending exposures and goals queued, so a later publish/1 or finalize/1 sends them again.

Finalizing

The finalize/1 function will ensure all events have been published to the ABsmartly collector, like publish/1, and will also "seal" the context, preventing any further events from being tracked.

:ok = Context.finalize(context)

true = Context.is_finalized?(context)

If publishing fails, finalize/1 returns {:error, reason} and the context stays open with its events queued, so finalize/1 can be called again.

finalize/1 keeps the context process alive so it can still be inspected. For request- or session-scoped contexts, call Context.dispose/1 instead: it finalizes (retrying a failed final publish up to :attempts times, default 3) and then always stops the process, releasing it and its retained state. If events could not be published it returns {:error, {:disposed_unpublished, reason}} so you can observe the loss; the documented integrations ignore the result because the process is released either way. Inspect the context (for example is_finalized?/1) before disposing, since the pid is dead afterwards.

Custom Event Logger

The ABsmartly SDK supports custom event logging for debugging, analytics, or integrating with other systems. Event loggers can be set at the SDK level or per-context.

Event Types

Event When Data
error Context data fails to load, or publish/1 / finalize/1 fails Error reason
ready Context turns ready ContextData used to initialize
refresh refresh/2 succeeds ContextData used to refresh
publish publish/1 succeeds Published events data
exposure treatment/2 succeeds on first exposure Exposure enqueued for publishing
goal track/2 or track/3 succeeds Goal enqueued for publishing
close finalize/1 succeeds the first time nil

Platform-Specific Examples

Using with Phoenix Framework

Integrate ABsmartly with Phoenix by creating a singleton GenServer and using it in your controllers.

# lib/my_app/absmartly_service.ex
defmodule MyApp.ABSmartlyService do
  use GenServer

  alias ABSmartly.SDK

  def start_link(_opts) do
    GenServer.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  def get_sdk do
    GenServer.call(__MODULE__, :get_sdk)
  end

  @impl true
  def init(:ok) do
    {:ok, sdk} = SDK.new(
      endpoint: Application.get_env(:my_app, :absmartly_endpoint),
      api_key: Application.get_env(:my_app, :absmartly_api_key),
      environment: Application.get_env(:my_app, :absmartly_environment),
      application: "my_app"
    )

    {:ok, %{sdk: sdk}}
  end

  @impl true
  def handle_call(:get_sdk, _from, state) do
    {:reply, {:ok, state.sdk}, state}
  end
end

# Add to application.ex supervision tree
defmodule MyApp.Application do
  use Application

  def start(_type, _args) do
    children = [
      MyAppWeb.Endpoint,
      MyApp.ABSmartlyService
    ]

    opts = [strategy: :one_for_one, name: MyApp.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

# lib/my_app_web/controllers/product_controller.ex
defmodule MyAppWeb.ProductController do
  use MyAppWeb, :controller

  alias ABSmartly.{SDK, Context}

  def show(conn, _params) do
    session_id = get_session(conn, :session_id) || generate_session_id()
    conn = put_session(conn, :session_id, session_id)

    {:ok, sdk} = MyApp.ABSmartlyService.get_sdk()

    units = %{"session_id" => session_id}

    case SDK.create_context(sdk, units) do
      {:ok, context} ->
        Context.set_attribute(context, "user_agent", get_req_header(conn, "user-agent"))

        treatment = Context.treatment(context, "exp_product_layout")

        Context.publish(context)
        Context.dispose(context)

        case treatment do
          0 -> render(conn, "show_control.html")
          _ -> render(conn, "show_treatment.html")
        end

      {:error, _reason} ->
        render(conn, "show_control.html")
    end
  end

  defp generate_session_id do
    :crypto.strong_rand_bytes(16) |> Base.encode16(case: :lower)
  end
end

Using with Phoenix LiveView

Handle experiments in LiveView with proper lifecycle management.

# lib/my_app_web/live/product_live.ex
defmodule MyAppWeb.ProductLive do
  use MyAppWeb, :live_view

  alias ABSmartly.{SDK, Context}

  def mount(_params, session, socket) do
    session_id = session["session_id"] || generate_session_id()

    {:ok, sdk} = MyApp.ABSmartlyService.get_sdk()
    units = %{"session_id" => session_id}

    case SDK.create_context(sdk, units) do
      {:ok, context} ->
        treatment = Context.treatment(context, "exp_product_layout")

        {:ok,
         socket
         |> assign(:context, context)
         |> assign(:treatment, treatment)}

      {:error, _reason} ->
        {:ok,
         socket
         |> assign(:context, nil)
         |> assign(:treatment, 0)}
    end
  end

  def handle_event("purchase", %{"amount" => amount}, socket) do
    if context = socket.assigns.context do
      Context.track(context, "purchase", %{"amount" => String.to_float(amount)})
      Context.publish(context)
    end

    {:noreply, socket}
  end

  def terminate(_reason, socket) do
    if context = socket.assigns.context do
      Context.dispose(context)
    end

    :ok
  end

  def render(assigns) do
    ~H"""
    <%= if @treatment == 0 do %>
      <div class="control-group">Control Layout</div>
    <% else %>
      <div class="treatment-group">Treatment Layout</div>
    <% end %>
    """
  end

  defp generate_session_id do
    :crypto.strong_rand_bytes(16) |> Base.encode16(case: :lower)
  end
end

Using with Plug

Create a Plug to automatically manage ABsmartly contexts for requests.

# lib/my_app_web/plugs/absmartly_context.ex
defmodule MyAppWeb.Plugs.ABSmartlyContext do
  import Plug.Conn

  alias ABSmartly.{SDK, Context}

  def init(opts), do: opts

  def call(conn, _opts) do
    session_id = get_session(conn, :session_id) || generate_session_id()
    conn = put_session(conn, :session_id, session_id)

    {:ok, sdk} = MyApp.ABSmartlyService.get_sdk()
    units = %{"session_id" => session_id}

    case SDK.create_context(sdk, units) do
      {:ok, context} ->
        user_agent =
          case get_req_header(conn, "user-agent") do
            [ua | _] -> ua
            [] -> "unknown"
          end

        Context.set_attribute(context, "user_agent", user_agent)

        conn
        |> assign(:absmartly_context, context)
        |> register_before_send(&finalize_context/1)

      {:error, _reason} ->
        conn
    end
  end

  defp finalize_context(conn) do
    if context = conn.assigns[:absmartly_context] do
      Context.publish(context)
      Context.dispose(context)
    end

    conn
  end

  defp generate_session_id do
    :crypto.strong_rand_bytes(16) |> Base.encode16(case: :lower)
  end
end

# Use in router
# lib/my_app_web/router.ex
defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
    plug MyAppWeb.Plugs.ABSmartlyContext
  end

  scope "/", MyAppWeb do
    pipe_through :browser

    get "/", PageController, :index
  end
end

Changelog

Assignment rules, semver operators and a versioned X-Agent

Added

  • Assignment rules. assign and split rules from an experiment's assignmentRules are applied (see Assignment Rules). An assignmentRules value that is not a JSON string, or not valid JSON, is ignored for that experiment only.
  • Semver operators. semver_eq, semver_gt, semver_gte, semver_lt and semver_lte for audience filters and rule conditions.
  • ABSmartly.HTTP.Client.agent/0 returns the X-Agent header value.

Changed

  • Exposures carry ruleOverride. ABSmartly.Types.Exposure (the :exposure event's data) has a new rule_override field, and every published exposure now has a ruleOverride key: true when a rule chose the variant, false for every other exposure (always sent, as in the JavaScript SDK).
  • X-Agent is versioned. The X-Agent header value sent on every request is now absmartly-elixir-sdk/<version> (e.g. absmartly-elixir-sdk/1.0.0) instead of absmartly-elixir-sdk, so the collector can report which SDK version sent each request.

Audiences always see application and environment

Before: audience filters saw only the attributes you set. The Web Console offers application and environment as attributes because the collector adds them to every event, but the SDK never had them, so a condition on either evaluated against a missing value and never matched.

After: the SDK always evaluates audiences with application and environment set to the names configured in the SDK. An attribute you set with the same name, at any time, overrides them. Names that are not configured or empty are left out. They are not published (the collector adds them server-side) and get_attributes/1 does not return them. The SDK never injects app_version (one you set yourself with Context.set_attribute(context, "app_version", ...) is still used and published like any other attribute).

This changes assignment for running experiments. A unit in an experiment with Audience enforced (audienceStrict) and an environment or application condition currently gets variant 0, even when it matches the condition and even when the experiment is full-on. After upgrading, a matching unit is randomised normally (or gets the full-on variant). Those experiments are broken today; this release fixes them, but their exposures will change. This SDK has never injected app_version, so audiences can only use an app_version value you set yourself (unchanged by this release).

About A/B Smartly

A/B Smartly is the leading provider of state-of-the-art, on-premises, full-stack experimentation platforms for engineering and product teams that want to confidently deploy features as fast as they can develop them.

A/B Smartly's real-time analytics helps engineering and product teams ensure that new features will improve the customer experience without breaking or degrading performance and/or business metrics.

Have a look at our growing list of clients and SDKs:

About

Elixir A/B testing SDK for cross-SDK tests

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages