Application Object Pooling

Beyond platform-provided pooling for messages and embedded entities, applications can use Talon's pooling facilities for custom objects to achieve zero-garbage operation.

Overview

Platform generated messages and entities generated with Xbuf encoding support pooling out of the box. Latency sensitive applications can also use the platform's pooling facilities for user objects. Using the platform's pooling facilities has several advantages:

• Pool usage is tracked and reported by XVM stats and heartbeats providing visibility into leaks

• Platform pools support preallocation out of the box

Using the platform's pooling facilities is considered an advanced feature of Talon. It is only recommended for users that have the most stringent latency and throughput requirements. A viable alternative to pooling may also be to use the low latency distribution of Talon which bundles Azul's Zing JVM which provides near pauseless GC. This is a lower complexity approach than pooling.

Coding Pooled Objects

Making an object poolable involves the following steps:

1. Implement com.neeve.util.UtlPool.Item

2. Create a UtlPool.Factory for creating new Items or arrays of Items (used by the pool to construct new instances)

Sample Object

The following example shows how to make an Order object poolable:

final public class Order implements Item<Order> {

    final private static class OrderFactory implements UtlPool.Factory<Order> {
        @Override
        final public Order createItem(final Object context) {
            return new Order();
        }

        @Override
        final public Order[] createItemArray(final int size) {
            return new Order[size];
        }
    }

    // TODO: your member variables
    private int orderQuantity;

    // order pool - member variable to store pool this Order belongs to
    private UtlPool<Order> pool;

    /**
     * Creates a new order pool with the provided number of preallocated orders.
     *
     * @param orderPreallocateCount The number of orders to preallocate in the pool
     * @param poolName The name of the pool
     * @return A new order pool
     */
    public static UtlPool<Order> createPool(int orderPreallocateCount, String poolName) {
        final UtlPool<Order> orderPool = UtlPool.create(
            "order",
            poolName,
            new OrderFactory(),
            UtlPool.Params.create()
                .setThreaded(false)
                .setInitialCapacity(orderPreallocateCount)
                .setPreallocate(true)
        );
        return orderPool;
    }

    private Order() {
        // initialization
        init();
    }

    /**
     * Implementation of {@link Item#init()}
     *
     * This method cleans a pool item when it is recycled to the pool or
     * added for the first time.
     */
    @Override
    final public Order init() {
        // TODO: reset your variables
        orderQuantity = -1;
        return this;
    }

    /**
     * Implementation of {@link Item#setPool}
     *
     * Called by the pool to mark that this instance belongs to it.
     */
    @Override
    final public Order setPool(UtlPool<Order> pool) {
        this.pool = pool;
        return this;
    }

    /**
     * Implementation of {@link Item#getPool}
     *
     * Gets the pool that this instance belongs to.
     */
    @Override
    final public UtlPool<Order> getPool() {
        return pool;
    }
}

Note the following:

• The UtlPool.Factory is implemented as a private inner class (though this is not mandatory)

• The Order object is a factory for its own pool via the static createPool() method (also not mandatory)

• The pool is created as non-threaded via UtlPool.Params passed in. This means only a single thread can take and/or put items into the returned pool. Because user code is single threaded, it is often acceptable to create pools as single threaded, particularly for preallocation use cases

• The Factory is created with 2 String parameters: the pool type and the pool name. The combination must be unique within the JVM

Implementing Reference Counting

The UtlPool.Item interface does not impose reference counting semantics, but you can add such behavior:

final public class Order implements Item<Order> {

    // SNIP - see the pooling code above

    // reference count for this object
    final protected AtomicInteger ownershipCount = new AtomicInteger(1);

    // order pool - member variable to store pool this Order belongs to
    private UtlPool<Order> pool;

    /**
     * Acquires a reference to the object.
     */
    @Override
    final public void acquire() {
        final int val = ownershipCount.incrementAndGet();
        if (val <= 1) {
            throw new IllegalStateException("attempt to acquire an already disposed Order!");
        }
    }

    /**
     * Returns the current ownership count of this object.
     */
    @Override
    final public int getOwnershipCount() {
        return ownershipCount.get();
    }

    /**
     * Implementation of {@link Item#init()}
     *
     * This method cleans a pool item.
     */
    @Override
    final public Order init() {
        // restore ownership count back to 1
        ownershipCount.getAndSet(1);
        // TODO: reset any other field values here
        return this;
    }

    /**
     * Disposes this order object. When its count drops to
     * 0 it will be returned to its pool (if pooled).
     */
    @Override
    public int dispose() {
        final int val = ownershipCount.decrementAndGet();
        if (val < 0) {
            throw new IllegalStateException("attempt to dispose an already disposed Order!");
        }

        if (val == 0 && pool != null) {
            pool.put(this);
        }
        return val;
    }
}

Using Pools

Once you have created your pooled object, you can create pools and use the objects:

UtlPool<Order> orderPool = Order.createPool(100, "order-pool");

Order order = orderPool.get(null); // ownershipCount is 1
// Do something
order.dispose(); // ownershipCount --> 0, object returned to pool

If the object will be passed off to another thread where it may be worked on in parallel, then you should acquire() a reference before transferring ownership:

Order order = orderPool.get(null); // ownershipCount is 1

// Hand off to a parallel thread
order.acquire(); // ownershipCount is 2
executor.execute(() -> {
    // Do stuff
    order.dispose();
});

// Do more things

order.dispose(); // ownershipCount --> 0? object returned to pool

Configuring Pools At Runtime

Note that config driven pool configuration is still in incubation and the following is subject to change.

In the pooled Order example above, the pool was hardcoded to preallocate a specific number of entries. It is possible to override programmatic configuration using environment variables. This can be achieved by setting properties of the form nv.pool.<poolKey>.<propertyName> where:

poolKey is: The <poolType>.<poolName> (the same as reported in pool stats without the trailing .instanceId suffix). In the example above this would be "order.order-pool"

propertyName is one of the bean properties on UtlPool.Params:

• initialCapacity

• maxCapacity

• threaded

• preallocate

• detachedWash

Pool properties can be configured in DDL as:

<env>
  <!-- pool parameters -->
  <nv>
    <pool>
      <!-- enable overrides of programmatically set values -->
      <overrideparamsfromenv>true</overrideparamsfromenv>

      <!-- configure pool of type "order" and name "order-pool" -->
      <order.order-pool>
        <initialCapacity>10000</initialCapacity>
        <preallocate>true</preallocate>
      </order.order-pool>
    </pool>
  </nv>
</env>

Pooling Configuration Properties

The following table summarizes pooling properties that can be set in env. Check the javadoc for UtlConstants for the most up to date values along with additional advanced properties.

Property Name	Default	Description
nv.pool.shouldpool	true	Property that controls whether pooling is enabled globally. Globally disabling pools is not usually recommended as it can have adverse consequences on memory management. This property is mainly provided for troubleshooting purposes.
nv.pool.sourceparamsfromenv	true	Property that controls whether pool parameters can be sourced from the environment. When true, calls to create pools will result in Params.load(poolkey, UtlEnv.getProps(), false) being invoked to apply pool parameters that haven't already been set explicitly. To override programmatically set values see nv.pool.overrideparamsfromenv. This setting is currently classified as experimental and is subject to change.
nv.pool.overrideparamsfromenv	false	Property that controls whether pool parameters can be overridden from the environment. By default, when pool parameters are sourced from the environment, they will not override values explicitly set programmatically. Setting this property forces the environment specified values to take precedence. This property should be used with extreme care as overriding programmatically set pool parameters can have adverse effects. This property takes no effect if nv.pool.sourceparamsfromenv is disabled. This setting is currently classified as experimental and is subject to change.

See Also

• Coding for Zero Garbage - Overview of zero garbage techniques

• Tuning Pools - Pool configuration and optimization

• Embedded Entities - Platform-provided pooled entities