Tuning Pools
To achieve zero garbage operation in steady state, Talon pools objects to avoid allocations. This section describes how to tune pools for optimal performance.
Overview
Pools created by the platform start out with no objects preallocated, so it is important for performance sensitive applications to drive warm-up traffic at startup not only to allow for JIT optimization to kick in, but also to make sure pools are grown to reach an equilibrium state where enough instances have been allocated to satisfy the number of objects in use at a given time.
To reach such an equilibrium state, applications should push traffic at rates higher than expected volume to ensure that pool sizes grow to a level that can accommodate spikes in traffic. In cases where application warm-up isn't sufficient for reaching pool equilibrium, pools can be manually configured.
Some cases where it is desirable to manually configure pool parameters include:
When performance (throughput and latency) is less important than memory footprint, disabling pools can avoid memory overhead where objects might otherwise sit in a pool unused
Cases where it is challenging to drive warm-up traffic to the point where pools reach optimal capacity - manual configuration with preallocation is desirable
Pools used purely in a preallocated fashion (e.g., applications that expect a given number of Order objects per day may want to start with those objects preallocated)
In some cases initial bursts of traffic at application startup cause pools of seldom used types to grow to a large size but then remain dormant taking up wasted memory - it would be desirable to limit the pool's capacity
General Pooling Configuration
Pooling of platform internal objects and embedded entities generated with Xbuf encoding is enabled by default. ADM Messages and State Entities can also be pooled when generated with Xbuf encoding, but pooling of these types is currently only enabled with nv.optimizefor=throughput or nv.optimizefor=latency. In certain cases it may be desirable to change this behavior at a more granular level.
nv.pool.shouldpool
true
This property can be set to false to globally disable all pooling including platform internal pools. Setting this value to false can adversely affect throughput and latency but can be useful in lowering memory overhead for lower performance use cases in which garbage collection costs are low.
nv.pkt.shouldpool
false (true when nv.optimizefor=throughput|latency)
This property globally enables or disables packet pooling. ADM generated Xbuf Messages or Entities are backed by and pooled with their backing packet objects, consequently this parameter controls pooling for these types. Note that embedded entities are not backed by a packet and are pooled independently of the Message or Entity in which they are contained, so this property does not affect embedded entities. Packet pooling impacts several areas of the platform including transaction logs and cluster replication as both operate using packets to frame their data.
Configuring Specific Pools
A pool is uniquely named in a JVM as <pooltype>.<poolname>.<poolinstanceid>. For example, a native 256 byte platform IOBuffer named "iobuf.native-256.23" has a pooltype of "iobuf", a pool name of "native-256" and is suffixed with a JVM unique instance id.
The following output from enabling pool stats trace shows some example pool names:
Pools are configured by specifying nv.pool.<poolidentifier>.<propertyname> where the identifier can be either the pooltype or the pooltype.poolname.
Pool Config Identifiers
Example pool identifiers from the above stats output:
"iobuf" - applies to all IOBuffer pools
"iobuf.native-256" - applies only to native 256 byte IOBuffer pools
"packet" - applies to all packet pools
"packet.MyMessageXbufPacket.71.1" - applies to the packet type backing the Xbuf generated MyMessage class with a factory id of 71 and type id of 1 (the pool instance id of 266 has no bearing on configuration and will change from run to run)
"xbuf.entity" - applies to all Xbuf embedded entities
"xbuf.entity.MyEntityXbufEntity.301.199" - applies to the embedded Xbuf MyEntity class with factory id 301 and type id 199
When both a type level and type+poolname property is configured, the higher granularity 'poolname' property takes precedence.
Pool Configuration Properties
The following environment variables can be used to configure a specific pool type using the above identifier:
nv.pool.<poolIdentifier>.initialCapacity
1024
The initial size of the array to hold pool items that are not in use.
nv.pool.<poolIdentifier>.maxCapacity
Integer.MAX_VALUE
The maximum number of slots for items not in use. If an item is returned to a pool and there are no slots left, it is evicted and becomes eligible for garbage collection.
nv.pool.<poolIdentifier>.threaded
true
Whether or not the pool will be safe for access by multiple threads.
nv.pool.<poolIdentifier>.preallocate
false
When true, pool is filled to its initial capacity with newly created items when the pool is created. When false, items are created on demand.
nv.pool.<poolIdentifier>.detachedWash
false
When items are returned to a pool, their fields are reset. When true, this makes items returned to the pool eligible for cleanup on a detached thread.
Note on above defaults: Each pool is created programmatically and may set its own default values; the above defaults apply to pools that haven't altered the default value.
DDL Pool Configuration Examples
The following configuration shows an example of configuring preallocation for the MyEntity Xbuf entity type:
See Also
Coding for Zero Garbage - Overview of zero garbage techniques
Application Object Pooling - Creating custom pooled objects
Configuring the Microservice Runtime - Runtime configuration options
Last updated