# Welcome to the X Platform

The X Platform is a platform to build, manage and monitor applications that process massive amounts of data in real-time. X does the heavy lifting of implementing all the infrastructure needs of such applications leaving the developer to focus solely on the domain and business logic. X applications are very easy to build with “zero-plumbing” business logic, can process massive amounts of live and historical data in real time with a no-compromise blend of extreme performance, total reliability and linear scale. X applications can be deployed, run, and managed on virtualized or bare metal infrastructure, in public or private clouds, on premise, on edge systems and hybrid environments using a single, unified deployment and management model and toolset. X Platform applications range from infrastructure products, such as content based message routers and complex event processing engines, to rich, functional, distributed applications, such as IoT event routers, financial services trading engines and eCommerce personalization engines.

The X Platform is comprised of the following set of modules.

## Talon

X Platform applications are micro-service based i.e. they are comprised of multiple microservices that collaborate with each other using message passing. Talon implements the core runtime of such microservices. The core purpose of Talon is ro provide a runtime that is "infrastructure complete". What this means is that the runtime implemented by Talon handles all the infrastructure needs of a microserice including but not limited to memory management, data storage and persistence, message encoding/decoding, journaling, message passing, linear scaling, high availability and consensus management. By completely taking care of the infrastructure needs of such data-intensive microservices, Talon leaves the developer to focus only on the domain and business logic of these microservices.

Talon is a closed source module that requires a license from Neeve.

## Eagle

Eagle is an X Platform module to build web applications. It layers on top of Talon and its applications are microservice based web applications.

Eagle is an open source module offered under the Apache 2.0 license.

## Hornet

Hornet is an X Platform module that implements a mechanism that maps message types to channels thus allowing for configuration driven registration of message interest registraton and channels for outbound send. It layers on top of Talon and also offers support for dependency injection based services.

Hornet is an open source module offered under the Apache 2.0 license.


# Set Up your Dev Environment

This section describes what needs to be done to set up your environment to develop X applications.

You will need the following to build and run X applications

* A JDK
* Maven
* An IDE
* An X License

## Install JDK

Ensure you have one of the following JDKs installed on your machine.

* JDK 8
* JDK 11
* JDK 17

You can download and install the JDK from [here](https://www.oracle.com/java/technologies/downloads/)

## Install Maven

X Platform modules implemements several plugins for code generation and preparation of deployment artifacts that integrate into the build cycle. Currently, all these plugins are built for Maven. Therefore, Maven is the preferred tool to build X Platform applications.

You can download and install Maven from [here](https://maven.apache.org).

Free Maven installation tutorials can be found at:

* [Windows](https://www.youtube.com/watch?v=Jtj-0yhox5s\&t=72s\&ab_channel=EvilTester-SoftwareTesting)
* [OSX](https://www.youtube.com/watch?v=EoXImdzlAls\&ab_channel=AutomationStepbyStep-RaghavPal)

Please make sure you install Maven v3.0.4 or later.

{% hint style="info" %}
**Using Other Build Tools:** The functionality of each Maven plugin implemented by the X Platform is also available as a CLI tool implemented by the core X runtime. In fact, the plugins and the CLI tools are each implemented using a common underlying API implemented by the X runtime. These CLI tools are intended for use with build tools other than Maven such as Gradle and Ivy
{% endhint %}

### Set JAVA\_HOME

Set the following in your environment

```
JAVA_HOME=<Home directory of the JDK which you would like to use to build X applications>
```

This setting is used by Maven to determine the JDK to use to build your X project.

### Building Using Java 11 or Beyond <a href="#building-using-java11-and-beyond" id="building-using-java11-and-beyond"></a>

To run X code generation plugins and build the generated core, you would need to include the following additional dependencies in your Maven project. This is due to the removal of these packages from the JDK after JDK8

```
<dependencies>
    <dependency>
        <groupId>javax.annotation</groupId>
        <artifactId>javax.annotation-api</artifactId>
        <version>1.3.2</version>
    </dependency>

    <dependency>
        <groupId>jakarta.xml.bind</groupId>
        <artifactId>jakarta.xml.bind-api</artifactId>
        <version>2.3.3</version>
    </dependency>

    <dependency>
        <groupId>org.glassfish.jaxb</groupId>
        <artifactId>jaxb-runtime</artifactId>
        <version>2.3.8</version>
    </dependency>
</dependencies>
 
```

{% hint style="info" %}
**Versions:** The versions above are the versions against which the X Platform has been tested.
{% endhint %}

## Get an X License

An X Platform license is require to run Talon based microservices. Please contact Neeve Sales or Neeve Support to obtain a license. You will receive the following information in the license package.

* The license file
  * Save that license to your home directory
* Credentials to the Neeve artifact repository
  * Update your Maven settings.xml file with the supplied credentials as described in the license package.

See [Working with Licenses](/get-started/working-with-licenses) for information on working with licenses.

{% hint style="info" %}
**The Neeve Artifact Repository:** The Neeve repository is the main repository for X build artifacts. The repository is intended for use by build tools such as Maven, Ivy, and Gradle to download X artifacts and by you to manually download artifacts.
{% endhint %}

## Setup your IDE

X applications can be developed using any IDE that supports developing of Java applications.


# Sanity Test your Environment

This section describes how to sanity test that your environment has been setup correctly to build and run X applications. We will do this using the Talon Starter Application which is included as one of the applications in the [samples repository](https://github.com/neeveresearch/nvx-samples).

{% hint style="info" %}
**Other Uses of the The Starter Application:** It is a good idea to download and run the starter application not only from the point of view of sanity testing your environment but also because we use this application to demonstrate how to use various features of an X application as well as to illustrate how non-functional aspects of the platform, such as configuration, horizontal scaling, message passing and high availability work in the platform.
{% endhint %}

## Download the Application

Execute the following to download the repository

```
git clone https://github.com/neeveresearch/nvx-samples
```

This will download the [samples repository](https://github.com/neeveresearch/nvx-samples) to a folder named `nvx-samples` relative to the directory from where the above command was executed.

{% hint style="info" %}
**Don't have Git on your local machine?:** If you do not have git installed on your local machine, go the [samples repository](https://github.com/neeveresearch/nvx-samples), click on the "Clone or Download" button and follow instructions to download the application to your machine.
{% endhint %}

The talon starter application is in the subdirectory `nvx-app-talon-starter`

## Build the Application

Execute the following to build all the sample applications including the starter application

```
cd nvx-samples
mvn -DskipTests clean install
```

The above will build the application and install it to your local maven repository.

{% hint style="info" %}
**Building with Java 11 or Beyond?** The sample application is implemented such that it can be built as is using Java 8. If you are building using Java 11 or beyond, you will need to add the extra dependencies specified in the [Building Using Java 11 and Beyond](/get-started/set-up-your-dev-environment#building-using-java11-and-beyond) section.
{% endhint %}

## The Starter Application

The starter aplication is comprised of three microservices

* Processor
* Receiver
* Sender

![19628218](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-6215b71baeca23aeac6b0ff7cdf4091bb2a8ee97%2F18382874.png?alt=media)

The Processor is a clustered, stateful, highly available and fault tolerant service. *Stateful* means it stores data that is durably persisted over time. *Fault tolerant* means that it will continue to operate without data or message loss in the event a resource failure. *Highly available* means that the MTTR to recovery on failure is minimal and *clustered* means that it implements high availability by means of hot standby clustering.

In the starter application, the Sender service sends messages which are received by the Processor. The Processor stores a count of messages it has received from the sender. Upon receipt of a message, the Processor increments its messages count in its store and sends another message. This message is received by the downstream Receiver microservice.

## Run the Application

This section describes how to run the starter application. A successful launch and run of the application effectively means that your environment is set up correctly to build and run X applications.

### Download and Install the Messaging Bus

X applications need a messaging bus for message passing betweeen the application's microservices. We use the ActiveMQ messaging bus with the starter application to sanity test your environment.

{% hint style="info" %}
**Support for Other Messaging Providers:** Out of the box, the X Platform supports several messaging bus providers such as Solace, JMS, ActiveMQ and Kafka. There is also a high performance messaging system being developed natively in the X Platform.
{% endhint %}

#### Install and Run ActiveMQ

[Download](https://activemq.apache.org/components/classic/download/) a binary distribution of ActiveMQ and unpack it into some directory. Then, type the following commands from the directory in which you have just unpacked the ActiveMQ distribution

```
cd bin
activemq
```

The ActiveMQ broker should launch and be ready ready for messaging..

### Launch the Starter Microservices

{% hint style="info" %}
**Running with Java 11 or Beyond?:** If you are using Java 11 or beyond to run the application, then please refer to the [Supported OS and Runtimes](/get-started/supported-os-and-runtimes) for instructions on Java modules to enable.
{% endhint %}

#### Start the Receiver

Execute the following from a *new* command shell

{% tabs %}
{% tab title="Linux/OSX" %}

```
$JAVA_HOME/java -Djava.net.preferIPv4Stack=true -cp "target/*:target/dependency/*" com.neeve.server.Main -n receiver 
-p desktop,activemq
```

{% endtab %}

{% tab title="Windows" %}

```
 %JAVA_HOME\java -Djava.net.preferIPv4Stack=true -cp "target\*;target\dependency\*" com.neeve.server.Main -n receiver 
-p desktop,activemq 
```

{% endtab %}
{% endtabs %}

You should see the trace similar to the following. This indicates that the receiver has successfully started.

```
...
<1,9705,My-MacBook-Pro.local> 20161101-07:34:25:476 (inf)...[AepEngine<receiver>] Engine started [Standalone, Non-Persistent, ICR Off].
<1,9705,My-MacBook-Pro.local> 20161101-07:34:25:476 (inf)...Server (NAME=receiver) startup complete.
.
.
.
<31,9705,My-MacBook-Pro.local> 20161101-07:34:25:500 (inf)...[AepEngine<receiver>] Messaging started. 
```

#### Start the Processor Cluster

Execute the following from a *new* command shell

{% tabs %}
{% tab title="Linux/OSX" %}

```
$JAVA_HOME/java -Djava.net.preferIPv4Stack=true -cp "target/*:target/dependency/*" com.neeve.server.Main -n processor-1 
-p desktop,activemq 
```

{% endtab %}

{% tab title="Windows" %}

```
%JAVA_HOME\java -Djava.net.preferIPv4Stack=true -cp "target\*;target\dependency\*" com.neeve.server.Main -n processor-1 
-p desktop,activemq 
```

{% endtab %}
{% endtabs %}

You should see the trace similar to the following. This indicates that the processor has successfully started.

```
<32,8456,My-MacBook-Pro.local> 20161030-16:22:59:083 (inf)...[RogLog->'processor'] Live transaction log file is 'processor.log'...
<32,8456,My-MacBook-Pro.local> 20161030-16:22:59:397 (inf)...Log preallocation (length=1073741824, mode=setLength) took 309 milliseconds.
<32,8456,My-MacBook-Pro.local> 20161030-16:22:59:405 (inf)...[RogLog->'processor'] Scavenging old log files....
<32,8456,My-MacBook-Pro.local> 20161030-16:22:59:405 (inf)...[RogLog->'processor'] ....scavenged 0 files (0 failed).
<1,8456,My-MacBook-Pro.local> 20161030-16:22:59:408 (inf)...[RogLog->'processor'] Materialized 0 entries (0 transactions) from the transaction log (in < 1 second).
<1,8456,My-MacBook-Pro.local> 20161030-16:22:59:409 (inf)...Initialization complete. Synchronized to transaction id #0 in the store.
<31,8456,My-MacBook-Pro.local> 20161030-16:22:59:710 (inf)...[AepEngine<processor>] Retransmitted 0 messages.
<1,8456,My-MacBook-Pro.local> 20161030-16:22:59:710 (inf)...[AepEngine<processor>] Engine started [Clustered, Primary, Persistent, ICR Off].
.
.
.
<31,8456,My-MacBook-Pro.local> 20161030-16:22:59:712 (inf)...[AepEngine<processor>] Messaging started.
```

The processor is a clustered service. Execute the following from a *new* command shell to start the second instance in the Processor cluster

{% tabs %}
{% tab title="Start the Cluster Backup (Linux/OSX)" %}

```
$JAVA_HOME/java -Djava.net.preferIPv4Stack=true -cp "target/*:target/dependency/*" com.neeve.server.Main -n processor-2 
-p desktop,activemq 
```

{% endtab %}

{% tab title="Start the Cluster Backup (Windows)" %}

```
%JAVA_HOME\java -Djava.net.preferIPv4Stack=true -cp "target\*;target\dependency\*" com.neeve.server.Main -n processor-2 
-p desktop,activemq 
```

{% endtab %}
{% endtabs %}

You should see trace similar to the following on the newly launched cluster instance:

```
<1,12030,My-MacBook-Pro.local> 20161102-23:59:46:258 (inf)...[AepEngine<processor>] Engine started [Clustered, Backup, Persistent, ICR Off].
```

and the trace similar to the following on the first instance

```
<31,11825,My-MacBook-Pro.local> 20161102-23:58:03:891 (inf)...[B->processor-11cdf912-f6a6-46c3-b74f-7578cf453652] Initializing member '4a2fd486-68da-47d5-95e6-b70ffab08086'...
<31,11825,My-MacBook-Pro.local> 20161102-23:59:46:231 (inf)...[B->processor-11cdf912-f6a6-46c3-b74f-7578cf453652] Member '4a2fd486-68da-47d5-95e6-b70ffab08086' initialization complete (transaction id=-1)...
<31,11825,My-MacBook-Pro.local> 20161102-23:59:46:231 (inf)...[RogLog->'processor'] Initialization complete. Synchronized to transaction id #20808 on the primary.
```

#### Start the Sender

Execute the following from a *new* command shell

{% tabs %}
{% tab title="Linux/OSX" %}

```
$JAVA_HOME/java -Djava.net.preferIPv4Stack=true -cp "target/*:target/dependency/*" com.neeve.server.Main -n sender -p 
desktop,activemq 
```

{% endtab %}

{% tab title="Windows" %}

```
%JAVA_HOME\java -Djava.net.preferIPv4Stack=true -cp "target\*;target\dependency\*" com.neeve.server.Main -n sender -p 
desktop,activemq 
```

{% endtab %}
{% endtabs %}

You should see the trace similar to the following. This indicates that the sender has successfully started.

```
<1,10110,My-MacBook-Pro.local> 20161101-09:30:18:827 (inf)...[AepEngine<sender>] Engine started [Standalone, Non-Persistent, ICR Off].
.
.
.
<31,10110,My-MacBook-Pro.local> 20161101-09:30:18:829 (inf)...[AepEngine<sender>] Messaging started.
Press Enter to send 10000 messages...
```

#### Run the Sender

The sender is programmed to wait for user input before sending any messages. Hit once you see the "Press Enter to send 10000 messages" message above. This will trigger the sending of 10k messages at 1k msgs/sec. You should see the following trace on the sender, processor, and receiver

{% tabs %}
{% tab title="Sender" %}

```
Press Enter to send 10000 messages...
Sent 1000 messages
Sent 2000 messages
.
.
.
Sent 10000 messages
Press Enter to send 10000 messages...
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Processor" %}

```
Processed 1000 messages
Processed 2000 messages
.
.
.
Processed 10000 messages
```

{% endtab %}
{% endtabs %}

{% tabs %}
{% tab title="Processor" %}

```
Received 1000 messages
Received 2000 messages
.
.
.
Received 10000 messages
```

{% endtab %}
{% endtabs %}

👏 Congratulations! You have just sanity tested your environment and succesfully built and run your first X application!


# Supported OS and Runtimes

This section list the runtimes and associated versions supported by the current version of the X Platform.

## Java

The current release of the X Platform has been run and tested on the following Java versions

<table><thead><tr><th width="102">Version</th><th width="122">Oracle JDK</th><th width="234">Azul Zulu (OpenJDK)</th><th>Azul Platform Prime (Zing)</th></tr></thead><tbody><tr><td>Java 8</td><td>jdk1.8.0_381</td><td>zulu8.74.0.17-ca-jdk8.0.392</td><td>zing23.09.0.0-7-jdk8.0.382</td></tr><tr><td>Java 11</td><td>jdk-11.0.20</td><td>zulu11.68.17-ca-jdk11.0.21</td><td>zing23.09.0.0-7-jdk11.0.20.1</td></tr><tr><td>Java 17</td><td>jdk-17.0.9</td><td>zulu17.46.19-ca-jdk17.0.9</td><td>zing23.09.0.0-7-jdk17.0.8.1</td></tr><tr><td>Java 21</td><td>jdk-21.0.8</td><td>zulu21.46.19-ca-jdk21.0.9</td><td>zing25.10.0.0-4-jdk21.0.8.0.101</td></tr></tbody></table>

### Running with Java 11 or Beyond

To run X Platform based applications using Java 11 and beyond, certain additional Java modules need to be opened. The following JVM params open the additional Java modules needed by the X Platform

```
--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED --illegal-access=warn
```

## Operating Systems

### Development

The X Platform is supported for development on any operating system supported by the above JVMs

### Production

The X Platform is supported for production on the following operating systems

<mark style="color:blue;">**Linux**</mark>

| Minimum Kernel Version | Minimum libc Version |
| :--------------------: | :------------------: |
|         3.10.0         |         2.17         |

### Test

This section lists the OS versions against which the various X Platform test suites have been run for this release.

**REGRESSION**

The X Platform *regression* test suite for this release has been run on the following operating systems

<mark style="color:blue;">**CentOS 7.9**</mark>

| Kernel Version | glibc Version |
| :------------: | :-----------: |
|     3.10.0     |      2.17     |

**SANITY**

The X Platform *sanity* test suite for this release has been run on the following operating systems

<mark style="color:blue;">**CentOS 7.9**</mark>

| Kernel Version | glibc Version |
| :------------: | :-----------: |
|     3.10.0     |      2.17     |

<mark style="color:blue;">**Amazon Linux 2023**</mark>

| Kernel Version | glibc Version |
| :------------: | :-----------: |
|     6.1.41     |      2.34     |

**PERFORMANCE**

The X Platform *performance* test suite for this release has been run on the following operating systems

<mark style="color:blue;">**CentOS 7.9**</mark>

| Kernel Version | glibc Version |
| :------------: | :-----------: |
|     3.10.0     |      2.17     |


# Working With Licenses

## Overview

To use the Talon runtime you must have a license. A license can be supplied to you by Neeve Sales or Neeve Support. Please contact them to obtain your free, evaluation or production license. You will be supplied with a license package containing the *xplatform.lic* license file, credentials to access the Neeve artifact repostitory, instructions on how to install the license and instructions on how to configure your build environment to use the artifact credentials.

## License Installation

You can install your license in your application's runtime environment and/or as an application resource and/or by placing it in the user home folder of the user running the JVM of the platform

### Installing your license in your home directory

Copy your xplatform.lic file to the home directory of the user that runs the X application(s).

If you are just starting with the X Platform, this can be the easiest option to get started.

### **Installing your license as an application resource**

Modify your application build to include xplatform.lic at the resource root i.e. /xplatform.lic (in either a jar or in your classes folder).

### Installing your license in your application's environment

1. Copy your license file to a disk location from which you will run the platform
2. Set one of the following environment variables or Java system properties to point to the *directory* containing license file.

* NVLICLOCATION
* nv.lic.location
* nv\_lic\_location

### A few points to note

1. The xplatform.lic file is a signed file. You must not modify the contents of xplatform.lic or it will be rendered invalid preventing your application from starting.
2. License enforcement is performed when using most libraries distributed with the platform. A failure to find a valid license file will result in System.exit(1) being called, which will terminate the JVM. 3. Precedence of locating a license file is as follows where the first license found is used:
   1. xplatform.lic located in the folder specified by System.getenv('NVLICLOCATION')
   2. xplatform.lic located in the folder specified by System.getenv('nv.lic.location')
   3. xplatform.lic located in the folder specified by System.getenv('nv\_lic\_location')
   4. xplatform.lic located in the folder specified by System.getProperty('NVLICLOCATION')
   5. xplatform.lic located in the folder specified by System.getProperty('nv.lic.location')
   6. xplatform.lic located in the folder specified by System.getProperty('nv\_lic\_location')
   7. xplatform.lic located in the user's home folder (given by System.getProperty("user.home"))
   8. '/xplatform.lic' on the application classpath
   9. A license bundled at a different path specific to a jar provided by Neeve.

{% hint style="info" %}
**xplatform.lic:** Note that at in all of the above cases the license location looks for a file named *xplatform.lic*. The file must not be renamed.
{% endhint %}

### Debugging License Resolution

To diagnose issues in license resolution you can set `-Dnv.license.debug=true` when launching the JVM. Setting this property will dump license resolution trace to `System.err` which will provide information on how your license was (or wasn't) loaded as well as the locations in which the above search steps are performed.

### When running a Talon microservice

When running a Talon microservice, your license information (including its expiration date if applicable) will be printed as part of the startup banner printed to System.out. The banner is only printed if the license was successfully located, otherwise the JVM will have exited before this information would be printed.

```
INFO   | jvm 1    | 2017/08/24 17:08:28 |    [License]
INFO   | jvm 1    | 2017/08/24 17:08:28 |      Issued To: Acme Co
INFO   | jvm 1    | 2017/08/24 17:08:28 |      Type     : Production
INFO   | jvm 1    | 2017/08/24 17:08:28 |      Expires  : Never
INFO   | jvm 1    | 2017/08/24 17:08:28 |      Source   : file:/root/xplatform.lic
```

## License Enforcement

The license enforcement check is done the first time one of the platform's license protected APIs is used. If the license is found to be invalid, the JVM is terminated after printing an error to System.err that will look something like the following:

```
****** X license verification failed "<error message>". Exiting... ******");
```

When running a Talon microservice in an XVM, the XVM will ensure the license check has been done prior to your application being started. If you are using X application libraries outside of an XVM, you may want to ensure that the license check has been completed by calling the [XRuntime.getLicense()](http://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html#getLicense\(\)) method at an appropriate point in your application lifecycle.

```
com.neeve.ci.XRuntime.getLicense()
```

If the license cannot be found, or the license has been tampered with, the process attempting to use the X Platform will be terminated. It is therefore important for applications that use the X Platform libraries outside of an XVM to ensure that the license is valid in an expeditious fashion.

### License Expiration Warnings

If your license has an expiration date, then starting 1 month before the expiration of your license you will see the following warning emitted to System.err on each startup of your application.

```
WARNING: your X Platform license will expire on <datetime>!
```


# Introduction

Talon provides the machinery to easily build, run and manage microservice based application systems. A microservice is a lightweight, stateful, fault tolerant and horizontally scalable message processor. It stores read-write state as Java objects in local memory and durably persisted in a fault tolerant manner and collaborates with other microservices using transactional, fire-n-forget message passing.

A Talon microservice is powered by the Talon runtime. Application developers implement business logic in message handlers. The Talon runtime uses configuration information to connect and subscribe to message streams of interest on a messaging fabric. It dispatches inbound messages to the application message handlers, the handlers execute business logic by invoking read-write operations on state and send outbound messages to downstream apps. Talon presents the state and messages to the microservice logic in the form of POJOs enabling the microservice logic to perform read-write operations on state and messages via simple getter and setter methods on the state/message objects. Talon ensures that these set/get operations execute at memory speeds, all microservice state changes are journaled in a replication manner and the application message handlers are invoked in an exactly once, fully fault tolerant and horizontally scalable manner.

## A Simple Microservice

Lets say, for example, a developer would like to write a microservice that performs the following

* Maintain a counter in its state
* Receive a message, update the counter with the value in the message and send an outbound message with the updated counter value

Furthermore, the developer needs to ensure that the system continues to operate without service interruption in the face of process, network, machine or data center failures. Talon makes it very simple to author such a microservice. To do so, a developer would do the following:

1. **Model the microservice data and messages in XML**: In the above microservice, the state is an object that contains a single counter field, the inbound message contains a field whose value needs to be added to the state counter and the outbound message contains a field with the updated value of the state counter.
2. **Inject the Talon code generator into the build process**: Application logic only works with POJOs and so the modelled state and messages need to be converted to POJOs. This is done using the Talon code generator. Talon integrates with build tools such as Maven, ANT and Gradle to enable the code generators to be injected into the appropriate stage in the build cycle.. At runtime, the microservice logic works with these generated objects, not the XML.
3. **Author a message handler:** The Talon runtime invokes the message handler on receipt of an inbound message. When invoked, Talon presents the handler with the store root object and the inbound message as POJOs. The handler reads the value from the inbound message, updates the counter field in its state and then creates, populates and sends an outbound message POJO populated with the updated counter value.

That's it. All the non-functional aspects of such a microservice, including lifecycle management, messaging connectivity, message encoding/decoding, in-memory data storage, message and state journaling and persistence, cluster replication and consensus management and linear scaling are all transparently taken care of by the Talon runtime.

### Microservice Data Store

A Talon microservice data store is modeled using XML converted to POJOs by the Talon code generator. The following is the model for the store for the above microservice. The model defines a state tree with a single root Repository object that contains a *long* field named *counter* that is used to hold the running counter.

```xml
<model>
    ...
    <entities>
        <entity name="Repository">
            <field name="counter" type="Long"/>
        </entity>
    </entities>
</model>
```

{% hint style="info" %}
**The Store**

The store of a Talon microservice is structured as an object tree. Each node of the tree can be an individual object or an object collection with the field type system equivalent to the Java type system. This enables arbitrarily complex state models.
{% endhint %}

{% hint style="info" %}
**State Replication vs Event Sourcing**

Talon supports two different types of microservices - State Replicated and Event Sourced. The above microservice is an example of a State Replication microservice that models state and uses the generated POJOs to store state. Event Sourced microservices, on the other hand, do not model state as XML and, instead, store their state in regular POJOs. This categorization is based on how the microservice manages consensus between the various microservice cluster members. State replication microservice establish cluster consensus by hot replicating changes to state to all cluster members while Event Sourced microservice establish consensus by replaying inbound messages on all cluster members.
{% endhint %}

### Application Messages

Talon messages are also modeled using XML converted to POJOs by the Talon code generator. The following is the model for the inbound and outbound messages processed by the above microservice.

```xml
<model>
    ...
    <messages>
        <message name="InMessage">
            <field name="value" type="Long"/>
        </message>

        <message name="OutMessage">
            <field name="total" type="Long"/>
        </message>
    </messages>
</model>
```

### Write Application Logic

From the point of view of an application developer, the business logic of a microservice is coded as message handlers. When a message arrives, the Talon runtime invokes the appropriate message handler, identified by message signature, with the message POJO and the POJO of the root object of the store. Between the inbound message and the state tree, the message handler has all that it needs to implement its business logic.

The below is the `InMessage` message handler for the above microservice.

```java
@EventHandler
public void onMessage(InMessage inMessage, Repository repository) {
    repository.setCounter(repository.getCounter() + inMessage.getValue());
    OutMessage outMessage = OutMessage.create();
    outMessage.setTotal(repository.getCounter());
    sender.sendMessage(“out”, outMessage);
}
```

That's it! Thats all the developer has to do to produce a fault tolerant and highly performant stateful message processor. In essence, Talon implements all the non-functional aspects of a stateful, message driven microservice leaving the developer to focus only on the business logic and domain. In doing so, it enables highly performance, fault tolerant and linearly scalable microservice based applications to be built, deployed and managed very easily.


# Concepts & Architecture

## Overview

Talon is a high-performance microservices platform designed for building ultra-low latency, fault-tolerant distributed systems. This section provides a comprehensive understanding of Talon's core concepts, architectural patterns, and operational models.

Understanding these concepts is essential for:

* **Architects** designing Talon-based systems
* **Developers** building Talon microservices
* **Operators** managing Talon deployments
* **Technical decision-makers** evaluating Talon for their use cases

## What You'll Learn

This section covers the foundational concepts and architectural patterns that underpin the entire Talon platform. Each topic builds on the others to provide a complete mental model of how Talon works.

### Core Architecture

[**Messaging Model**](/talon/concepts-and-architecture/messaging-model) - How microservices communicate

Talon's messaging abstraction provides a unified interface for message exchange across different messaging backbones (Solace, JMS, etc.). Learn how:

* Messages are modeled as strongly-typed POJOs
* Channel subscriptions and message routing work
* Different messaging bindings integrate with the platform
* Message delivery guarantees are provided

[**Microservice Architecture**](/talon/concepts-and-architecture/microservice-architecture) - The anatomy of a Talon microservice

Understand the structure of a Talon microservice:

* [Runtime Architecture](/talon/concepts-and-architecture/microservice-architecture/runtime-architecture) - The runtime components (AEP Engine, SMA, XVM, etc.)
* [Development Model](/talon/concepts-and-architecture/microservice-architecture/development-model) - Development artifacts (ADM models, user code, configuration)
* [Configuration Model](/talon/concepts-and-architecture/microservice-architecture/configuration-model) - How configuration is managed and applied

### Operational Concepts

[**Microservice Operation**](/talon/concepts-and-architecture/microservice-operation) - How microservices behave at runtime

Follow a microservice through its complete lifecycle:

* [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Complete microservice lifecycle with all phases and events
* [Initialization](/talon/concepts-and-architecture/microservice-operation/cluster-initialization) - How microservices start up and initialize
* [Cluster Join](/talon/concepts-and-architecture/microservice-operation/cluster-join) - How instances discover and join clusters
* [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) - The message processing loop and transaction flow
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How distributed consensus is achieved
* [Cluster Failover](/talon/concepts-and-architecture/microservice-operation/cluster-failover) - Handling primary failures and backup promotion

### High Availability & Consistency

[**Consensus Models**](/talon/concepts-and-architecture/consensus-models) - State Replication vs Event Sourcing

Talon provides two consensus models for achieving high availability:

* **Event Sourcing** - Replicate inbound messages for deterministic state reconstruction
* **State Replication** - Replicate state changes automatically

Understand the trade-offs between latency, complexity, and flexibility.

[**Transactions**](/talon/concepts-and-architecture/transactions) - ACID guarantees in distributed systems

Learn how Talon provides transactional guarantees:

* Atomic commits of state changes and outbound messages
* Transaction isolation and consistency
* Integration with consensus models
* Savepoints and transaction control

### Platform Services

[**Threading Model**](/talon/concepts-and-architecture/threading-model) - How Talon manages concurrency

Talon's single-threaded business logic model eliminates race conditions while maintaining high throughput:

* Thread architecture and responsibilities
* Disruptor-based event processing
* Thread affinitization for performance
* NUMA-aware configuration

[**Discovery Model**](/talon/concepts-and-architecture/discovery-model) - How components find each other

The discovery service enables microservices to locate:

* Other microservice instances for clustering
* XVM containers for deployment
* Message buses for connectivity
* Discovery providers (Multicast, SMA, Local)

[**Operating Model**](/talon/concepts-and-architecture/operating-model) - Administration, monitoring, and troubleshooting

Understand the operational aspects:

* Statistics collection and heartbeats
* Administrative commands and control
* Analysis and troubleshooting tools
* Performance monitoring

## Architecture Principles

Talon's architecture is guided by several key principles:

### 1. Event-Driven Processing

All business logic executes in response to events (messages, lifecycle events, timer events). This ensures deterministic, reproducible behavior across instances.

### 2. Single-Threaded Business Logic

Message handlers execute on a single thread, eliminating the need for locks, mutexes, or concurrent data structures in application code.

### 3. Transparent High Availability

Clustering, replication, and failover happen automatically without application code changes. The platform handles distributed consensus.

### 4. Message-Oriented Architecture

Communication between microservices happens exclusively through messages. This loose coupling enables independent deployment and scaling.

### 5. Strong Typing

All messages and state are strongly-typed POJOs generated from declarative models. This eliminates serialization errors and provides compile-time safety.

### 6. Configuration as Code

All runtime configuration is expressed in declarative XML (DDL) with support for templating, profiles, and overrides.

## How These Concepts Fit Together

Here's how the concepts relate to building and running a Talon microservice:

1. **Development**: Use the [Development Model](/talon/concepts-and-architecture/microservice-architecture/development-model) to understand what artifacts you create (ADM models, handlers, configuration)
2. **Runtime**: The [Runtime Architecture](/talon/concepts-and-architecture/microservice-architecture/runtime-architecture) shows how those artifacts are loaded and executed by the AEP Engine, SMA, and XVM
3. **Communication**: The [Messaging Model](/talon/concepts-and-architecture/messaging-model) explains how your microservice sends and receives messages via the SMA
4. **Lifecycle**: [Microservice Operation](/talon/concepts-and-architecture/microservice-operation) describes what happens from startup through steady-state processing to shutdown
5. **High Availability**: [Consensus Models](/talon/concepts-and-architecture/consensus-models) and [Transactions](/talon/concepts-and-architecture/transactions) explain how consistency and fault tolerance are achieved
6. **Performance**: The [Threading Model](/talon/concepts-and-architecture/threading-model) and [Discovery Model](/talon/concepts-and-architecture/discovery-model) describe how Talon achieves ultra-low latency
7. **Operations**: The [Operating Model](/talon/concepts-and-architecture/operating-model) covers monitoring, administration, and troubleshooting

## Next Steps

After understanding the concepts in this section, proceed to:

* [**Developing Applications**](/talon/developing-applications) - Put these concepts into practice by building microservices
* [**Microservice Template**](/talon/developing-applications/microservice-template) - Start with template projects for Event Sourcing or State Replication
* [**Reference**](/talon/reference) - Look up specific annotations, events, and configuration elements

## Additional Resources

For hands-on learning:

* Start with [Get Started](/get-started/set-up-your-dev-environment) to set up your development environment
* Review [Introduction](/talon/introduction) for a high-level overview of Talon
* Explore the [Microservice Template](/talon/developing-applications/microservice-template) section for complete implementation examples


# Messaging Model

Talon microservices are message-driven services that communicate through a flexible and powerful messaging model built on the Simple Messaging API (SMA). SMA provides a messaging abstraction layer that allows applications to communicate with one another (and external applications) using a consistent API, regardless of the underlying messaging provider.

## Overview

SMA defines a simple yet robust messaging API that allows any **bus provider** (messaging backbone such as Solace, JMS brokers, etc.) to be seamlessly integrated into the platform by implementing a **messaging binding** for that provider. The AEP engine underlying a Talon microservice handles interaction with SMA on behalf of the microservice, managing the lifecycle of messaging connections, providing send methods, and dispatching received messages to message handlers in a transactionally atomic fashion with respect to the microservice's state updates.

At a high level, developers model messages using the Application Data Modeler (ADM) and map those messages to logical message buses and channels. Through configuration, logical buses are bound to actual messaging fabric implementations provided by bus providers (e.g., a Solace broker URL), and message channels are mapped to topics on that messaging fabric.

*SMA provides an abstraction layer between Talon microservices and messaging providers*

## Core Messaging Abstractions

### Message Buses

A **message bus** groups messaging participants (bus users) that exchange messages. Each bus serves as a messaging sandbox containing one or more message channels. An AEP engine uses its name as the bus username when connecting to a message bus.

### Message Bus User

A **bus user** is a messaging participant. When an AEP engine connects to a bus, it identifies itself using a username (typically the engine's configured name). This identity is used for:

* Connection authentication and authorization
* Message sender identification
* Administrative operations

### Messages (Views)

A **message** (represented by the [`MessageView`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageView.html) interface) is the unit of exchange between messaging participants. A MessageView is a Plain Old Java Object that:

* Implements the `MessageView` interface
* Provides accessors to the message's underlying fields
* Handles encoding and decoding for transmission
* Is modeled using the Application Data Modeler (ADM)

Messages are typed objects generated from ADM schema definitions. See [Modeling Messages & State](/talon/developing-applications/modeling-messages-and-state) for details on message modeling.

### Message Encoding

SMA transports MessageViews as bytes encoded according to the message's encoding type. The platform natively supports:

* **JSON** - Human-readable text format for debugging and interoperability
* **Protobuf** - Google Protocol Buffers for efficient binary encoding
* **Xbuf2** - Talon's zero-garbage implementation with Protobuf wire encoding for maximum performance

The platform also has limited support for **Custom** encodings, useful for integrating with formats not native to the platform. Custom encoding is currently available in custom message bus bindings, with broader support anticipated.

See [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type) for guidance on selecting an encoding, and [Understanding Message Serialization](/talon/concepts-and-architecture/messaging-model/understanding-message-serialization) for the encoding type values as they appear on the wire.

### Message Channels

A **message channel** serves as a named conduit for messages between messaging participants. Senders send messages on a channel, and receivers express interest by joining the channel. Each channel can be configured with:

* **Quality of Service (QoS)**: BestEffort or Guaranteed delivery
* **Channel Keys**: Routing patterns that map channels to provider destinations

Channels are mapped to physical message bus provider destinations (such as topics) via a **channel key**, which allows channels to be bound to different message providers through configuration.

*Channels are mapped to provider destinations via channel keys*

#### Joining and Leaving Channels

Specifying that a channel should be **joined** indicates interest in receiving messages on that channel. When a channel is configured for join, the bus provider issues subscriptions for the associated topic. When a bus channel is closed, the microservice can elect to **leave** the channel, removing previously issued subscriptions.

See [Registering Message Interest](/talon/developing-applications/configuring-messaging/registering-message-interest) for configuration details.

#### Channel Quality of Service

SMA supports two qualities of service for message channels:

* **BestEffort**: In the absence of failures in the microservice or message fabric, messages won't be dropped. No durability guarantees across failures.
* **Guaranteed**: The messaging provider must support at-least-once delivery of messages even in the presence of message fabric or microservice failures. An AEP engine working with Guaranteed QoS supports exactly-once processing by filtering out duplicates, provided the message bus provider provides appropriate ordering guarantees.

### Channel Keys and Filters

**Channel keys** and **channel filters** enable fine-grained message routing on the bus:

* **Channel Keys**: Configured bus-wide and define a routing pattern (e.g., `Orders/${Region}/${Firm}/${Symbol}`)
* **Message Keys**: Resolved at send time using the channel key and message contents via **Key Resolution Tables (KRT)**
* **Channel Filters**: Configured per participant to specify which messages to receive (e.g., `Region=US|EMEA;Firm=BOA`)

A sent message is delivered only to participants with filters that match the message's resolved key.

#### Static vs Dynamic Keys

Channel keys can be either **static** or **dynamic**:

* **Static Keys**: Fixed topic names that don't vary (e.g., `OrderEvents`)
* **Dynamic Keys**: Allow substitution of variable portions in the topic at runtime, with dynamic components sourced from fields on the message being sent or from channel filters

**Example**: A message sent on a channel with a dynamic key of `Orders/${Region}/${Firm}/${Symbol}` where the message has:

* `getRegion() == "EMEA"`
* `getFirm() == "BOA"`
* `getSymbol() == "MSFT"`

Would be sent on topic: `Orders/EMEA/BOA/MSFT`

#### Channel Filters

A channel filter is used to filter subscriptions issued for a channel that has a dynamic key.

**Example**: A channel joined with:

* Dynamic key: `Orders/${OrderState}/${Region}`
* Filter: `OrderState=New|Canceled;Region=US`

Would issue subscriptions for:

* `Orders/New/US`
* `Orders/Canceled/US`

See [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) for details on message keys and routing.

### Message Participation

To participate in message exchange, a Talon microservice:

1. **Connects** to one or more message buses
2. **Joins** channels it's interested in receiving messages from
3. **Processes** inbound messages in message handlers
4. **Sends** outbound messages through channels

The AEP engine manages the lifecycle and policies of messaging connections and registers message interest on behalf of the microservice.

## Inbound Channel Resolution

Everything above describes the send side. When a message *arrives*, the binding has to decide which channel to dispatch it on, and that decision is where most surprises live.

The framing to hold on to is this:

> **The topic decides whether a message arrives. The channel decides how it is dispatched once it has arrived.**

The topic (the resolved channel key) is what the subscription was issued for, so it governs whether the message reaches the microservice at all. Once it has arrived, the topic is finished, it plays no part in choosing the channel. Channel resolution uses only the channel identity carried in the message metadata.

### Channel Identity on the Wire

Message metadata carries two pieces of channel identity: a **channel id** and a **channel name**. A channel does not send the same identity on every message.

* The **first** message sent on a channel carries **both** the name and the id.
* After that first send, the name is dropped **only if the channel's id is valid**. Subsequent messages then carry the id alone.
* If the channel's id is **not** valid, the name is never dropped, and every message from that channel carries its **name** and never a usable id.

That last case is worth reading twice, because it is the explanation for an observation that looks alarming: a channel with no valid id sends `ChannelID=-1` on every single message, forever, and everything still works. The channel is being identified by name.

### What Counts as a Valid Id

An id is valid when it is **greater than zero**.

So `-1` and `0` both mean *no id was sent*. They are not wrong ids, and they are not error codes. A channel name length of `-1` means the same thing for the name: absent.

### The Resolution Table

Given what arrived, the binding resolves as follows:

| Name present | Valid id present | What happens                                                                                                                                                               |
| ------------ | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Yes          | Yes              | Resolve by name. If the id on the message does not match the local channel's id, log a warning and **blacklist that id**, but still dispatch on the name-resolved channel. |
| Yes          | No               | Resolve by name.                                                                                                                                                           |
| No           | Yes              | Resolve by id, unless that id has been blacklisted.                                                                                                                        |
| No           | No               | Nothing to resolve with.                                                                                                                                                   |

Blacklisting exists because a mismatched id is genuinely ambiguous. The receiver has no idea what channel the sender meant by it, so the id is not trusted again. The name is unambiguous, so the message is still delivered.

### The Catch-All Channel

If resolution produced no channel, the binding makes one more attempt: it looks for a **catch-all channel**, a channel with the reserved id `32767`.

Two things about this are surprising, and both are load-bearing:

* A channel that **matched by name but is not joined** also falls through to the catch-all. "Found" is not sufficient; it must be joined.
* The catch-all is used **regardless of its own join state**.

The catch-all is **off by default** (`auto_add_catchall_channel` defaults to `false`). Enabling it is a real behavioural change, not a safety net.

{% hint style="warning" %}
Do not enable a catch-all channel by default. With it off, a message that resolves to nothing is reported as unhandled, and you find out. With it on, that same message is silently dispatched on the catch-all instead. You are trading a visible failure for an invisible one, which is only the right trade if you have a specific reason to want it.
{% endhint %}

### When Nothing Resolves

If there is no channel and no catch-all, the binding raises an error that surfaces as an SMA `UnhandledMessageEvent`.

It never becomes an AEP `MessageEvent`, which means **no `@EventHandler` runs**. From the application's point of view the message simply never arrived. If you are debugging a message that was definitely published but never processed, this is the path to check, see [Unhandled Messages](/talon/developing-applications/authoring-user-code/message-processing/unhandled-messages).

### Why There Is No Channel-Less Send

Every `AepEngine.sendMessage` overload takes a `MessageChannel`. The channel is what supplies the bus binding, the QoS, and key resolution. A topic supplies none of those, which is why sending on a topic instead of a channel does not exist at any layer. It is not a restriction that was imposed, it is that a topic does not carry enough information to send with.

For the wire-level detail of how channel id and name are carried, see [Understanding Message Serialization](/talon/concepts-and-architecture/messaging-model/understanding-message-serialization).

## Messaging Bindings

A **messaging binding** (also called a "bus binding") is the component that connects a Talon microservice to the messaging fabric provided by a **bus provider** (the underlying messaging backbone such as Solace, ActiveMQ, Tibco EMS, etc.). The binding handles:

* **Protocol Translation**: Converting between message POJOs (Plain Old Java Objects implementing the MessageView interface) and the provider's native protocol
* **Connection Management**: Establishing and maintaining connections to the messaging infrastructure
* **Message Transport**: Sending and receiving messages via the provider's API
* **Provider-Specific Features**: Leveraging unique capabilities of each messaging technology

### Binding Architecture

When a messaging user, such as the AEP engine, sends a message POJO (Plain Old Java Object implementing the MessageView interface) through a message bus binding:

1. The message POJO is serialized using a particular encoding (JSON, Protobuf, or Xbuf2)
2. The binding prepares a metadata object that contains metadata information, including information for the receiving side to know how to reconstitute the message POJO from the serialized form
3. The binding transmits the serialized message via the provider's native protocol
4. On the receiving side, the binding receives the provider's native message format
5. The binding uses the metadata information to deserialize the message and reconstruct the message POJO (MessageView instance)
6. The binding dispatches the message POJO to the receiving user

### Connection Descriptors

Each binding is configured through a **connection descriptor** that specifies:

* **Provider Type**: Which binding to use (jms, solace, loopback, executor)
* **Connection Details**: Address, port, credentials
* **Provider-Specific Properties**: Settings unique to each binding
* **Common Properties**: Cross-provider settings (key handling, concurrent sends, etc.)

Example descriptor formats:

```xml
<!-- Non-decomposed format -->
<bus name="orders-bus" descriptor="activemq://localhost:61616&set_key_on_receipt=true" />

<!-- Decomposed format -->
<bus name="orders-bus">
  <provider>activemq</provider>
  <address>localhost</address>
  <port>61616</port>
  <properties>
    <set_key_on_receipt>true</set_key_on_receipt>
  </properties>
</bus>
```

## Available Bindings

Talon provides built-in bindings for various messaging technologies:

### [Solace Binding](/talon/concepts-and-architecture/messaging-model/solace-binding)

Native integration with Solace PubSub+ message brokers using JCSMP (Java) or CCSMP (JNI) for high-performance, low-latency messaging.

**Use when:** Ultra-low latency and high throughput requirements with Solace PubSub+ infrastructure

**Configuration:** [Solace Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/solace-binding)

### [JMS Binding](/talon/concepts-and-architecture/messaging-model/jms-binding)

JNDI-based integration with JMS 1.1 message brokers, including provider-specific optimizations for ActiveMQ and Tibco EMS.

**Use when:** Integrating with existing JMS infrastructure or enterprise messaging systems

**Configuration:** [JMS Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/jms-binding)

### [Loopback Binding](/talon/concepts-and-architecture/messaging-model/loopback-binding)

In-memory messaging for applications running in the same process.

**Use when:** Development, testing, or single-process deployments

**Configuration:** [Loopback Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/loopback-binding)

### [Executor Binding](/talon/concepts-and-architecture/messaging-model/executor-binding)

Thread-based messaging for offloading processor-intensive work or implementing outbound gateways.

**Use when:** Processor-intensive tasks need to run on separate threads or implementing custom outbound integrations

**Configuration:** [Executor Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/executor-binding)

## Choosing a Binding

Consider these factors when selecting a messaging binding:

* **Existing Infrastructure**: Use bindings compatible with your deployed messaging systems
* **Performance Requirements**: Match latency and throughput needs to binding capabilities
* **Deployment Model**: Cloud, on-premise, hybrid, or single-JVM
* **Message Persistence**: Whether messages need to survive restarts
* **Scalability**: Single-process vs. distributed system requirements

## Related Topics

* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - How to configure buses and bindings
* [Registering Message Interest](/talon/developing-applications/configuring-messaging/registering-message-interest) - How to join channels and receive messages
* [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - How to send messages through channels
* [Understanding Message Serialization](/talon/concepts-and-architecture/messaging-model/understanding-message-serialization) - Wire format, message metadata, and integrating non-Talon applications

## Next Steps

1. Review individual binding pages to understand each provider's characteristics
2. Learn how to [configure bus connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections)
3. Understand [message processing](/talon/developing-applications/authoring-user-code/message-processing) in microservices


# JMS Binding

The JMS binding provides integration with JMS 1.1 message brokers using JNDI-based configuration for Talon microservices.

## Overview

The JMS binding works with JMS 1.1 level JMS clients and is configured using JNDI lookup. In addition to the generic JNDI-based configuration, the platform provides provider-specific implementations optimized for:

* **ActiveMQ** - Uses `INDIVIDUAL_ACKNOWLEDGE_MODE` for Guaranteed delivery
* **Tibco EMS** - Uses `EXPLICIT_CLIENT_ACKNOWLEDGE` for Guaranteed delivery

Both provider-specific implementations normalize topic delimiters from `/` to `.` for consistency with other Talon bindings.

For configuration details, see [JMS Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/jms-binding).

## JMS Topic Format

JMS topics are JMS provider-specific. The platform treats channel key levels as being delimited by `/` and leaves it up to the binding implementation to normalize topic levels to the format native to the JMS provider.

For example, the ActiveMQ and Tibco EMS providers both normalize the channel key separator to `.` when sending or joining a channel.

## Wildcard Topics

Support for topic wildcards is specific to the JMS provider, if supported at all. The generic JMS binding does not perform any special handling for wildcard characters. When `nv.sma.cleanchannelfilter=true`, all non-alphanumeric characters are replaced with an `_` character (as they are during message sends when `nv.sma.cleanmessagekey=true`).

The platform's ActiveMQ and Tibco EMS specific binding implementations are sensitive to the following wildcards when `nv.sma.cleanchannelfilter=true` and preserve these characters rather than replace them with `_`:

| Wildcard | Description                                                                                  | Example                                                                                                                                                                                                                                            |
| -------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*`      | Matches 0 or more characters within a topic level. Must be the only character in that level. | <p><code>orders/\*</code> matches:<br>- <code>orders/gin</code><br>- <code>orders/begin</code><br>- <code>orders/ginseng</code><br>- <code>orders/beginning</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>orders/in</code></p> |
| `...`    | Matches multiple topic levels. May only be used as the last level in a topic filter string.  | <p><code>orders/...</code> matches:<br>- <code>orders/events/MSFT</code><br>- <code>orders/updates/APPL</code><br>- <code>orders/cancels/US/IBM</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>order-updates</code></p>         |

## Sending and Receiving from External Applications

For the metadata field semantics and wire layout these examples rely on, see [Understanding Message Serialization](/talon/concepts-and-architecture/messaging-model/understanding-message-serialization).

This section describes integration with non-Talon applications over JMS. This is an uncommon use case - see [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) for Talon-to-Talon messaging.

{% hint style="info" %}
These examples use ADM-generated messages for convenience, but this is not required. Messages can be encoded in any format (e.g., Protobuf, Xbuf) and encoded/decoded manually.
{% endhint %}

### Sending Messages to Talon

The platform's JMS binding encodes a serialized message payload in a bytes message and sets metadata fields in its JMS headers.

**Example using JMS client**:

```java
// populate a message
OrderEventMessage orderEvent = OrderEventMessage.create();
orderEvent.setOrderId(1);

// serialize the payload to a byte buffer
byte[] serializedPayload = orderEvent.serializeToByteArray();

// prepare JMS message
javax.jms.Message message = null;
if (orderEvent.getEncodingType() == MessageView.ENCODING_TYPE_JSON) {
  message = session.createTextMessage();
  ((TextMessage)message).setText(new String(serializedPayload));
}
else {
  message = session.createBytesMessage();
  ((BytesMessage)message).writeBytes(serializedPayload);
}

// prepare message metadata
MessageMetadata metadata = MessageMetadataFactory.getInstance().createMessageMetadata();
metadata.serializeV2(orderEvent.getMessageEncodingType(),
                     orderEvent.getVfid(),
                     orderEvent.getType(),
                     0,  // message sender id
                     0,  // message flow
                     0,  // message sequence number (unsequenced)
                     -1, // channel id (unspecified)
                     XString.create("order-events"));

// The JMS binding decomposes metadata into message properties
message.setBooleanProperty(JmsMessageBusBinding.SMA_METADATA_PRESENT_JMSPROP, true);
message.setByteProperty(JmsMessageBusBinding.SMA_METADATA_VERSION_JMSPROP, (byte)metadata.getVersion());
message.setByteProperty(JmsMessageBusBinding.SMA_METADATA_ENCODING_JMSPROP, metadata.getMessageEncodingType());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_VFID_JMSPROP, metadata.getMessageViewFactory());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_VTYPE_JMSPROP, metadata.getMessageViewType());
message.setIntProperty(JmsMessageBusBinding.SMA_METADATA_SENDER_JMSPROP, metadata.getMessageSender());
message.setIntProperty(JmsMessageBusBinding.SMA_METADATA_FLOW_JMSPROP, metadata.getMessageFlow());
message.setLongProperty(JmsMessageBusBinding.SMA_METADATA_SNO_JMSPROP, metadata.getMessageSno());
message.setShortProperty(JmsMessageBusBinding.SMA_METADATA_CHID_JMSPROP, metadata.getMessageChannelId());
message.setStringProperty(JmsMessageBusBinding.SMA_METADATA_CHNAME_JMSPROP, metadata.getMessageChannelName());

// send
Topic topic = session.createTopic("order-events");
messageProducer.publish(topic, message, DeliveryMode.PERSISTENT, 0, 0);

// dispose
orderEvent.dispose();
metadata.dispose();
```

### Receiving Messages from Talon

**Example using JMS client**:

```java
public void onMessage(javax.jms.Message message) {
  // extract SMA metadata
  final boolean isMetadataPresent = message.getBooleanProperty(SMA_METADATA_PRESENT_JMSPROP);
  if (!isMetadataPresent) {
    // not a Talon message
    handleNonTalonMessage(message);
    return;
  }

  final byte encodingType = message.getByteProperty(SMA_METADATA_ENCODING_JMSPROP);
  final short vfid = message.getShortProperty(SMA_METADATA_VFID_JMSPROP);
  final byte version = message.getByteProperty(SMA_METADATA_VERSION_JMSPROP);
  final short vtype = version > MessageMetadata.V1 ? message.getShortProperty(SMA_METADATA_VTYPE_JMSPROP) : 0;

  // extract payload
  Object payload = null;
  if (message instanceof TextMessage) {
    payload = ((TextMessage) message).getText();
  }
  else if (message instanceof BytesMessage) {
    BytesMessage bytesMessage = (BytesMessage) message;
    byte[] bytes = new byte[(int) bytesMessage.getBodyLength()];
    bytesMessage.readBytes(bytes);
    payload = bytes;
  }
  else {
    handleNonTalonMessage(message);
    return;
  }

  // lookup factory (assumes view factories already registered)
  MessageViewFactory factory = viewFactoryRegistry.getMessageViewFactory(vfid);
  if (factory != null) {
    MessageView view = factory.wrap(vtype, encodingType, payload);
    if (view instanceof OrderEventMessage) {
      handleOrderEvent((OrderEventMessage) view);
    }
  }
  else {
    handleNonTalonMessage(message);
  }
}
```

## See Also

* [JMS Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/jms-binding) - Configuration reference
* [Messaging Model](/talon/concepts-and-architecture/messaging-model) - Overview of Talon messaging
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration


# Solace Binding

The Solace binding provides native integration with Solace PubSub+ message brokers for high-performance, low-latency messaging in Talon microservices.

## Overview

The Solace binding uses Solace's native APIs (not JMS) to achieve optimal performance with Solace PubSub+ message brokers. It supports both Solace PubSub+ hardware appliances and software brokers.

For configuration details, see [Solace Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/solace-binding).

## JNI vs Java Implementation

The Solace binding includes both a Java-based and JNI-based implementation:

### Java Binding (JCSMP)

* Uses Solace's **JCSMP** (Java Client Software Messaging Protocol) API
* Works on **all platforms** supported by Talon
* **Not zero-garbage** but provides good performance
* Fully managed Java implementation

### JNI Binding (CCSMP)

* Uses Solace's **CCSMP** (C Client Software Messaging Protocol) API via JNI
* **Linux only**
* Supports **zero-garbage messaging** in steady state
* Maximum performance for latency-sensitive applications

### Key Differences

| Aspect                         | Java (JCSMP)                     | JNI (CCSMP)                                    |
| ------------------------------ | -------------------------------- | ---------------------------------------------- |
| Platforms                      | All                              | Linux only                                     |
| Garbage                        | Non-zero                         | Zero in steady state                           |
| UnhandledMessageEvent topic    | Sent topic name                  | Subscription name                              |
| UnhandledMessageEvent sequence | Appliance-wide sequence number   | Session-specific sequence number               |
| Pass-through properties        | Properties starting with `jcsmp` | Properties starting with `FLOW_` or `SESSION_` |

{% hint style="info" %}
The binding rationalizes some common properties between the two implementations, but if using properties specific to JCSMP or CCSMP that aren't rationalized, consider whether JNI is enabled.
{% endhint %}

## Session Lifecycle

Both implementations expose the broker session through `com.neeve.solxf.ISolSession`. The interface declares `connect()` to establish the session and `disconnect()` to tear it down. `disconnect()` is idempotent, so disconnecting an already disconnected session is a no-op.

{% hint style="warning" %}
`disconnect()` was added to `ISolSession` in Solace 3.16.40. The bindings supply it, so normal use needs no action on upgrade. A custom implementation of `ISolSession`, if one exists, must provide the method before it will compile against this release.
{% endhint %}

## Solace Topic Format

Solace topics are hierarchical with each level separated by `/` character. Topics cannot exceed 250 characters in length.

**Example topics**:

* `orders/events/MSFT`
* `market-data/trades/US/APPL`
* `alerts/critical/region1`

For more details on Solace topics, refer to [Solace documentation](https://docs.solace.com/Features/SMF-Topics.htm).

## Wildcard Topics

Solace supports wildcard characters for matching multiple topics in [channel filters](/talon/developing-applications/configuring-messaging/registering-message-interest). Wildcards are **not** applied to sent topics (treated as literals).

The Solace binding preserves these wildcards when `nv.sma.cleanchannelfilter=true`:

| Wildcard | Description                                                          | Example                                                                                                                                                                                                                                  |
| -------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*`      | Matches 0 or more characters at end of topic level                   | <p><code>orders/gin\*</code> matches:<br>- <code>orders/gin</code><br>- <code>orders/ginseng</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>orders/in</code><br>- <code>orders/gin/foo</code></p>                     |
| `>`      | Matches multiple topic levels. Must be only character in last level. | <p><code>orders/></code> matches:<br>- <code>orders/events/MSFT</code><br>- <code>orders/updates/APPL</code><br>- <code>orders/cancels/US/IBM</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>order-updates</code></p> |

### Topic Routing Example

The following example demonstrates dynamic topic resolution with wildcards:

```xml
<buses>
  <bus name="trading" descriptor="solace://solacehost:55555&topic_starts_with_channel=false">
    <channels>
      <channel name="orders">
        <qos>Guaranteed</qos>
        <key>${InitiatingClient::NONE}/${Region::NONE}/${Symbol::NONE}</key>
      </channel>
    </channels>
  </bus>
</buses>

<apps>
  <app name="order-manager">
    <messaging>
      <buses>
        <bus name="trading">
          <channels>
            <channel name="orders">
              <filter>Region="US";Symbol=A*|MSFT</filter>
            </channel>
          </channels>
        </bus>
      </buses>
    </messaging>
  </app>

  <app name="client">
    <messaging>
      <buses>
        <bus name="trading" join="true">
          <channels>
            <channel name="orders" join="false" />
          </channels>
        </bus>
      </buses>
    </messaging>
  </app>
</apps>
```

The order-manager application uses filter `Region="US";Symbol=A*|MSFT`, creating these subscriptions:

* `*/US/A*` - Any client, US region, symbols starting with A
* `*/US/MSFT` - Any client, US region, MSFT symbol

**Matching messages**:

```
{InitiatingClient: "CLIENTA", Region: "US", Symbol: "MSFT"} → CLIENTA/US/MSFT → matches */US/MSFT
{InitiatingClient: "CLIENTA", Region: "US", Symbol: "APPL"} → CLIENTA/US/APPL → matches */US/A*
{Region: "US", Symbol: "MSFT"} → NONE/US/MSFT → matches */US/MSFT
```

**Non-matching messages**:

```
{InitiatingClient: "CLIENTA", Region: "US", Symbol: "IBM"} → CLIENTA/US/IBM → no match
{Symbol: "IBM"} → NONE/NONE/IBM → no match
```

## Message Metadata Version

When messages are sent through the Solace binding, SMA (Simple Messaging API) metadata is included to help receivers deserialize and dispatch messages.

By default, the Solace binding sends messages using **V1 metadata** to ensure compatibility with older receivers (pre-Talon 1.8.396). The difference:

* **V1**: Receivers must introspect serialized message content to determine type
* **V2**: Message type is encoded in metadata (more efficient)

{% hint style="info" %}
If all downstream receivers use Talon 1.8.396 or later, set `sma_metadata_version=2` as a Solace binding property for more efficient metadata.
{% endhint %}

## Sending and Receiving from External Applications

For the metadata field semantics and wire layout these examples rely on, see [Understanding Message Serialization](/talon/concepts-and-architecture/messaging-model/understanding-message-serialization).

This section describes integration with non-Talon applications over Solace. This is an uncommon use case - see [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) for Talon-to-Talon messaging.

{% hint style="info" %}
These examples use ADM-generated messages for convenience, but this is not required. Messages can be encoded in any format (e.g., Protobuf, Xbuf) and encoded/decoded manually.
{% endhint %}

### Sending Messages to Talon

The Solace binding transports encoded messages in a `BytesMessage` with:

* **Message data**: Serialized message payload
* **x-sma-metadata property**: Serialized `MessageMetadata`

**Example using Solace JCSMP client**:

```java
// populate a message
OrderEventMessage orderEvent = OrderEventMessage.create();
orderEvent.setOrderId(1);

// serialize the payload to a byte buffer
byte[] serializedPayload = orderEvent.serializeToByteArray();

// prepare message metadata
MessageMetadata metadata = MessageMetadataFactory.getInstance().createMessageMetadata();
metadata.serializeV2(orderEvent.getMessageEncodingType(),
                     orderEvent.getVfid(),
                     orderEvent.getType(),
                     0,  // message sender id
                     0,  // message flow
                     0,  // message sequence number (unsequenced)
                     -1, // channel id (unspecified)
                     XString.create("order-events"));
byte[] serializedMetadata = metadata.serializeToByteArray();

// prepare solace message
com.solacesystems.jcsmp.BytesMessage message = producer.createBytesMessage();
message.setData(serializedPayload);
message.setDeliveryMode(DeliveryMode.DIRECT);
SDTMap props = producer.createMap();
if (serializedMetadata != null) {
  props.putBytes("x-sma-metadata", serializedMetadata);
}
message.setProperties(props);

// send
com.solacesystems.jcsmp.Topic topic = producer.createTopic("order-events");
producer.send(message, topic);

// dispose
orderEvent.dispose();
metadata.dispose();
```

### Receiving Messages from Talon

**Example using Solace JCSMP client**:

```java
public void onMessage(BytesXMLMessage message) {
  if (!(message instanceof BytesMessage)) {
    handleNonTalonMessage(message);
    return;
  }

  BytesMessage bytesMessage = (BytesMessage) message;

  // extract sma metadata
  final SDTMap props = bytesMessage.getProperties();
  ByteBuffer serializedMetadata = null;
  if (props != null) {
    byte[] metadataBytes = props.getBytes("x-sma-metadata");
    if (metadataBytes != null) {
      serializedMetadata = ByteBuffer.wrap(metadataBytes);
    }
  }

  if (serializedMetadata == null) {
    handleNonTalonMessage(message);
    return;
  }

  // extract metadata fields
  final byte encodingType = MessageMetadata.getMessageEncodingType(serializedMetadata);
  final short vfid = MessageMetadata.getMessageViewFactory(serializedMetadata);
  final short vtype = MessageMetadata.getMessageViewType(serializedMetadata);

  // extract payload
  byte[] payloadBytes = bytesMessage.getData();
  Object payload = payloadBytes;
  if (encodingType == MessageView.ENCODING_TYPE_JSON) {
    payload = new String(payloadBytes);
  }

  // lookup factory (assumes view factories already registered)
  MessageViewFactory factory = viewFactoryRegistry.getMessageViewFactory(vfid);
  if (factory != null) {
    MessageView view = factory.wrap(vtype, encodingType, payload);
    if (view instanceof OrderEventMessage) {
      handleOrderEvent((OrderEventMessage) view);
    }
  }
  else {
    handleNonTalonMessage(message);
  }
}
```

## See Also

* [Solace Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/solace-binding) - Configuration reference
* [Messaging Model](/talon/concepts-and-architecture/messaging-model) - Overview of Talon messaging
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration
* [Solace PubSub+](https://solace.com/products/message-routers) - Solace message broker information


# Loopback Binding

The Loopback binding enables in-process message exchange between Talon microservices for testing and development.

## Overview

The platform's Loopback binding can be used to send messages to other applications running in the **same process**, not across different processes. This binding is often used for unit testing in which several applications may be launched in the same process for easy debugging.

When you configure a bus binding to use a loopback bus, a LoopbackBus instance is statically created on demand using the address provided by the binding. Two applications that connect to a loopback bus with the same name may exchange messages with one another.

For configuration details, see [Loopback Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/loopback-binding).

{% hint style="warning" %}
Loopback buses work **only within a single process**. They cannot be used for communication between different processes.
{% endhint %}

## Loopback Bus Topic Format

Loopback bus topics can be specified hierarchically with each level in the topic separated by a `/` character. A valid loopback topic may start with a `/`, but it is not required.

**Example topics**:

* `orders/events`
* `/market-data/trades`
* `alerts/critical/region1`

## Wildcard Topics

The loopback bus supports 2 wildcard character sequences that can be used in [channel filters](/talon/developing-applications/configuring-messaging/registering-message-interest) to match multiple sending topics. Wildcards are **not** applied to sent topics (treated as literals).

The Loopback binding preserves these wildcards when `nv.sma.cleanchannelfilter=true`:

| Wildcard | Description                                                                             | Example                                                                                                                                                                                                                                                                   |
| -------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `*`      | Matches 0 or more characters within a topic level                                       | <p><code>orders/</code><em><code>gin</code></em> matches:<br>- <code>orders/gin</code><br>- <code>orders/begin</code><br>- <code>orders/ginseng</code><br>- <code>orders/beginning</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>orders/in</code></p> |
| `...`    | Matches multiple topic levels. Must be used as the last level in a topic filter string. | <p><code>orders/...</code> matches:<br>- <code>orders/events/MSFT</code><br>- <code>orders/updates/APPL</code><br>- <code>orders/cancels/US/IBM</code><br><br>Does not match:<br>- <code>orders</code><br>- <code>order-updates</code></p>                                |

## Use Cases

The Loopback binding is primarily used for:

* **Unit Testing** - Test multiple microservices in a single process without requiring external message infrastructure
* **Development** - Simplify debugging by running multiple applications in the same JVM
* **Integration Testing** - Test microservice interactions without network dependencies
* **Configuration Flexibility** - Use property substitution to default to loopback for testing but switch to production message buses at runtime

## See Also

* [Loopback Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/loopback-binding) - Configuration reference
* [Messaging Model](/talon/concepts-and-architecture/messaging-model) - Overview of Talon messaging
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration


# Executor Binding

The Executor binding enables applications to offload work to separate threads for processor-intensive tasks or outbound gateway implementations.

## Overview

The Executor Bus Binding is a special bus binding that allows applications to provide send work (via a message) to be processed on a separate thread. The executor binding can be used to:

* Perform processor-intensive work in a thread other than an application's main dispatch thread
* Implement outbound gateways where the executing thread 'pushes' the sent message to an external system

Work done by the processor of an executor bus is acknowledged and therefore Guaranteed across failures.

For configuration details, see [Executor Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/executor-binding).

{% hint style="warning" %}
The Executor Bus is still in incubation and is classified as an experimental feature. The APIs may change as new features are added to this binding.
{% endhint %}

## Architecture

```
+------------------+
|   Application    |
|  (Main Thread)   |
+--------+---------+
         | send()
         v
+------------------+
|  Executor Bus    |
|  (Detached       |
|   Send Queue)    |
+--------+---------+
         | process()
         v
+------------------+
|   Processor      |
|   (Worker        |
|    Thread)       |
+--------+---------+
         |
         v
+------------------+
|   External       |
|   System         |
|  (e.g., SMTP)    |
+------------------+
```

## How It Works

To use an executor bus, the application must:

1. **Implement an ExecutorBusProcessor** that handles the processing
2. **Expose it via an ExecutorBusProcessorFactory** configured for the bus
3. **Acknowledge completed work** via the provided Acknowledger as work is completed
4. **Send the processor work** in the form of messages that the processor will complete

## Implementing an Executor Bus Processor

An executor bus needs an [`ExecutorBusProcessor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/ExecutorBusProcessor.html) to perform processing, which is supplied to the bus when it is created by its executor bus processor factory.

### The Process Method

The `ExecutorBusProcessor` interface defines a single method:

```java
/**
 * Called by the executor bus to process a message sent through the executor bus.
 * <p>
 * The implementer may perform the send in the thread calling this method or
 * pass the message off to a thread pool that it manages for greater parallelism.
 * <p>
 * Users of an executor bus should be able to expect ordered processing of messages
 * on a per channel basis, so implementations that perform processing on multiple
 * threads are encouraged to call MessageView.getMessageChannel() to determine
 * the execution channel and process accordingly.
 *
 * @param view The view to send.
 * @param acknowledger The acknowledger or null if no acknowledgement is required for processing this message.
 * @param flags Flags provided by the executor bus as hints to the processor.
 */
public void process(MessageView view, Acknowledger acknowledger, int flags) throws Exception;
```

### Lifecycle Integration

Processors that need to open connections to external systems may implement [`LifecycleAwareExecutorBusProcessor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/LifecycleAwareExecutorBusProcessor.html) to hook into the bus lifecycle:

* `onExecutorBusOpen()` - Called when the bus is opened
* `onExecutorBusStart()` - Called when the bus is started
* `onExecutorBusClose()` - Called when the bus is closed

### Accessing Configuration Properties

Processor-related configuration properties from the bus descriptor can be retrieved from the provider config portion of the binding's descriptor:

```java
@Override
public void onExecutorBusOpen(MessageBusBinding binding) throws Exception {
    Properties config = binding.getDescriptor().getProviderConfig();
    String smtpHost = config.getProperty("smtp_host");
    // ... configure processor
}
```

### Acknowledger

Regardless of whether or not an executor bus channel is configured to be Guaranteed, the bus will pass a non-null Acknowledger to the application and the application must call its `acknowledge()` method when processing has been completed.

The acknowledger can be called by any thread asynchronously, but its `acknowledge()` method may only be called once as the Acknowledger implementation is pooled.

## Sample Implementation

The following pseudo-code illustrates how an executor bus processor can be implemented:

```java
public class EmailGatewayProcessor implements LifecycleAwareExecutorBusProcessor {

    private final Tracer tracer = Tracer.create("email.sender", INFO);
    private volatile JavaMailSenderImpl mailer;

    /**
     * Called by executor bus prior to open.
     */
    @Override
    public void onExecutorBusOpen(MessageBusBinding binding) throws Exception {
        // Get config properties:
        Properties config = binding.getDescriptor().getProviderConfig();
        mailer = new JavaMailSenderImpl();
        mailer.setHost(config.getProperty("smtp_host"));
        mailer.setPort(Short.valueOf(config.getProperty("smtp_port")));
        mailer.setUsername(config.getProperty("smtp_user"));
        mailer.setPassword(config.getProperty("smtp_password"));
        mailer.setProtocol(config.getProperty("smtp_protocol"));
        // etc...
        tracer.log("EmailSender opened", INFO);
    }

    /**
     * Called by executor bus binding on start.
     */
    @Override
    public void onExecutorBusStart(MessageBusBinding binding) throws Exception {
        tracer.log("EmailSender started", INFO);
    }

    /**
     * Called by executor bus binding on close.
     */
    @Override
    public void onExecutorBusClose(MessageBusBinding binding) throws Exception {
        mailer = null;
        tracer.log("EmailSender closed", INFO);
    }

    /**
     * Executor bus callback.
     */
    @Override
    public final void process(MessageView message, Acknowledger acknowledger, int flags) {
        try {
            final MimeMessage template = mailer.createMimeMessage();
            final MimeMessageHelper helper = new MimeMessageHelper(template, true);

            helper.setSubject("ALERT: " + message.getClass().getSimpleName());
            helper.setText(message.serializeToJson(), false);
            // etc...

            // send
            mailer.send(template);

            // acknowledge completion:
            acknowledger.acknowledge();
        }
        catch (Exception e) {
            // Acknowledge with a failure (to close the bus):
            acknowledger.acknowledge(e);
        }
    }
}
```

## Implementing a Processor Factory

The processor factory returns instances of a processor for use by the executor bus. The executor bus will create a processor when the bus is created. In an AEP Engine, this will be when an engine is activated.

```java
/**
 * Factory for creating EmailGatewayProcessors
 */
public class EmailGatewayProcessorFactory extends AbstractExecutorBusProcessorFactory {
    @Override
    public ExecutorBusProcessor createExecutorBusProcessor(MessageBusBinding binding) {
        return new EmailGatewayProcessor();
    }
}
```

{% hint style="info" %}
**Executor Bus Processor Dependencies**

If the class implementing ExecutorBusProcessor is created through a DI framework or needs access to other objects in your application, consider registering the instance of your processor as a static variable in the bus processor factory. If you have multiple processor instances, you can store them in a static map and use the bus binding descriptor to determine which instance to return.

Useful binding configuration includes:

* **binding.getUsername()**: The name of the engine creating the bus
* **binding.getName()**: The name of the bus as configured in DDL
* **binding.getAddress()**: The address as configured in DDL
* **binding.getDescriptor().getProperties()**: Binding properties as configured in DDL
  {% endhint %}

## Example: E-mail Alert Gateway

This example demonstrates creating a gateway that bridges alerts received from Solace out through an email gateway.

### Application Code

```java
@AppHAPolicy(value = HAPolicy.StateReplication)
public class EmailGatewayApp {
    private volatile AepMessageSender messageSender;

    @AppInjectionPoint
    public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    @EventHandler
    public void onAlert(MyAppAlertMessage alert) {
        // Send a copy of the received alert out through the
        // email-alerts channel of the e-mail bus:
        messageSender.send("email-alerts", alert.copy());
    }
}
```

### Configuration

```xml
<buses>
  <bus name="alert-bus">
    <provider>solace</provider>
    <address>localhost</address>
    <port>55555</port>
    <channels>
      <channel name="app-alerts">
        <qos>Guaranteed</qos>
      </channel>
    </channels>
  </bus>
  <bus name="email-sender">
    <provider>executor</provider>
    <address>audit-logger</address>
    <properties>
      <processor_factory_classname>com.example.EmailGatewayProcessorFactory</processor_factory_classname>
      <from_address>admin@example.com</from_address>
      <smtp_host>mail.example.com</smtp_host>
      <smtp_port>25</smtp_port>
      <smtp_password>admin</smtp_password>
      <!-- ... and so on ... -->
    </properties>
    <channels>
      <channel name="email-alerts">
        <qos>Guaranteed</qos>
      </channel>
    </channels>
  </bus>
</buses>

<apps>
  <app name="email-gateway-app" mainClass="com.example.EmailGatewayApp">
    <messaging>
      <buses>
        <bus name="alert-bus">
          <detachedSend enabled="false"/>
          <channels>
            <channel name="app-alerts" join="true"/>
          </channels>
        </bus>
        <bus name="email-sender">
          <detachedSend enabled="true">
            <queueDrainerCpuAffinityMask>${EMAIL_SENDER_CPU_AFFMASK::0}</queueDrainerCpuAffinityMask>
          </detachedSend>
          <channels>
            <channel name="email-alerts" join="false"/>
          </channels>
        </bus>
      </buses>
    </messaging>
  </app>
</apps>
```

Because the e-mail gateway bus is acknowledging its work after sending each e-mail, this application will guarantee that e-mail alerts will be sent, and by virtue of clustering will be highly available.

## See Also

* [Executor Binding Configuration](/talon/developing-applications/configuring-messaging/configuring-bus-connections/executor-binding) - Configuration reference
* [Messaging Model](/talon/concepts-and-architecture/messaging-model) - Overview of Talon messaging
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration
* [`ExecutorBusProcessor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/ExecutorBusProcessor.html) - Processor interface
* [`LifecycleAwareExecutorBusProcessor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/LifecycleAwareExecutorBusProcessor.html) - Lifecycle interface
* [`AbstractExecutorBusProcessorFactory`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/AbstractExecutorBusProcessorFactory.html) - Factory base class


# Understanding Message Serialization

Most Talon applications never think about how a message becomes bytes. You declare a message in a model, call `sendMessage()`, and a handler on another microservice receives the same message back as a Java object. The bus binding does the encoding on the way out and the decoding on the way in.

This page is for the cases where that is not enough:

* You are integrating an application that does not run on Talon, and it has to produce or consume Talon messages directly.
* You are looking at bytes in a wire sniffer, a broker's message browser, or a log, and you need to know what they mean.
* You are writing a custom bus binding, or a tool that reads a message log.

## Serialization and Deserialization

Talon messages are ordinary Java objects. A message declared in an ADM model generates a class implementing `MessageView`, and that interface carries the full serialization surface.

Two things travel on the wire for every message:

* The **message payload**, the encoded fields of the message itself.
* The **message metadata**, a small fixed-layout header that tells the receiver how to interpret the payload, and which channel the message was sent on.

The payload alone is not enough to reconstruct a message. The receiver needs the metadata to know which encoding was used and which generated factory and type to hand the bytes to. How the two are carried is a property of the binding: the Solace binding puts the payload in the message body and the metadata in an `x-sma-metadata` property; the JMS binding does the same over a `BytesMessage`.

## Engine-Independent Serializers

`com.neeve.sma.MessageView` exposes serialization that does not require an engine, a binding, or any running Talon infrastructure. Given a message object you can turn it into bytes, and given bytes you can turn them back into a message.

The methods come in families, one per destination type:

| Destination       | Serialize                                                    | Deserialize                                                             |
| ----------------- | ------------------------------------------------------------ | ----------------------------------------------------------------------- |
| `byte[]`          | `serializeToByteArray()`, `serializeTo(byte[], int)`         | `deserializeFromByteArray(byte[])`, `deserializeFrom(byte[], int, int)` |
| `ByteBuffer`      | `serializeToByteBuffer()`, `serializeTo(ByteBuffer)`         | `deserializeFromByteBuffer(ByteBuffer)`, `deserializeFrom(ByteBuffer)`  |
| `IOBuffer`        | `serializeToIOBuffer(boolean)`, `serializeTo(IOBuffer, int)` | `deserializeFrom(IOBuffer, int, int)`                                   |
| `IOElasticBuffer` | `serializeTo(IOElasticBuffer, int)`                          | `deserializeFrom(IOElasticBuffer, int, int)`                            |
| `PktPacket`       | `serializeToPacket()`, `serializeTo(PktPacket)`              | `deserializeFromPacket(PktPacket)`, `deserializeFrom(PktPacket)`        |
| Native address    | `serializeTo(long, int)`                                     | `deserializeFrom(long, int, int)`                                       |
| JSON              | `serializeToJson()`                                          | `deserializeFromJson(String)`                                           |

Note the **offset-into-existing-buffer** overloads, `serializeTo(byte[] array, int offset)` and friends. These write into a buffer you already own rather than allocating a new one, and they return the number of bytes written. If you are assembling a larger frame, or you are on a path where allocation matters, these are the ones to reach for; the `serializeToXxx()` forms allocate.

{% hint style="info" %}
`serializeToJson()` and `deserializeFromJson()` are a convenience for debugging and interoperability. They are not the same thing as the JSON *encoding type* below, which is the wire encoding negotiated for a channel.
{% endhint %}

See the [`MessageView`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageView.html) Javadoc for exact signatures and per-method semantics.

## Message Metadata

[`MessageMetadata`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageMetadata.html) is the header that accompanies every message. It has a fixed binary layout, which is what makes it readable by an application that is not running Talon.

### Message Encoding Type

The encoding type is a single byte identifying how the payload was encoded:

| Value | Constant                 | Encoding |
| ----- | ------------------------ | -------- |
| 3     | `ENCODING_TYPE_XBUF`     | Xbuf     |
| 4     | `ENCODING_TYPE_PROTOBUF` | Protobuf |
| 5     | `ENCODING_TYPE_JSON`     | JSON     |
| 7     | `ENCODING_TYPE_XBUF2`    | Xbuf2    |

There is no value 2.

{% hint style="warning" %}
The legacy Confluence page listed `3 = Protobuf` and `4 = Xbuf`. Those two are **the wrong way round**, and the page predates Xbuf2 entirely. The table above is taken from the `ENCODING_TYPE_*` constants in `MessageView`. If you have code or notes derived from the old page, check this field.
{% endhint %}

For new applications, prefer Protobuf, Xbuf2, JSON, or Custom. Xbuf is effectively superseded by Xbuf2. See [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type).

### Field Reference

| Field               | Meaning                                                                                    |
| ------------------- | ------------------------------------------------------------------------------------------ |
| Version             | Metadata wire format version (1 or 2). Determines the layout of everything after it.       |
| Encoding type       | How the payload is encoded. See the table above.                                           |
| View factory        | Id of the ADM-generated factory that can create the message.                               |
| View type           | Id of the message type within that factory. **V2 only**. On V1 metadata this reads as `0`. |
| Sender              | Id of the sending member. `0` when unspecified.                                            |
| Flow                | Message flow id, used for ordering and duplicate detection. `0` when unspecified.          |
| Sno                 | Message sequence number within the flow. `0` for an unsequenced message.                   |
| Channel id          | Id of the channel the message was sent on, or a non-positive value if no id was sent.      |
| Channel name length | Length in bytes of the channel name that follows, or `-1` if no name was sent.             |
| Channel name        | The channel name, if present. Variable length.                                             |

The view factory and view type together identify the message class. This is why a V1 receiver has to introspect the payload to work out what it is holding, while a V2 receiver can dispatch straight from the metadata.

{% hint style="info" %}
Channel id and channel name are the inputs to inbound channel resolution, and a message does not necessarily carry both. See [Inbound Channel Resolution](/talon/concepts-and-architecture/messaging-model#inbound-channel-resolution) for what the receiver does with them, and why a channel id of `-1` is normal rather than an error.
{% endhint %}

### V1 Wire Layout

Fixed portion is 24 bytes, followed by the variable-length channel name.

| Offset | Size | Field                                |
| ------ | ---- | ------------------------------------ |
| 0      | 1    | Version (`1`)                        |
| 1      | 1    | Encoding type                        |
| 2      | 2    | View factory                         |
| 4      | 4    | Sender                               |
| 8      | 4    | Flow                                 |
| 12     | 8    | Sno                                  |
| 20     | 2    | Channel id                           |
| 22     | 2    | Channel name length (`-1` if absent) |
| 24     | *n*  | Channel name                         |

### V2 Wire Layout

V2 inserts the view type after the view factory, shifting everything below it by two bytes. The fixed portion is 26 bytes.

| Offset | Size | Field                                |
| ------ | ---- | ------------------------------------ |
| 0      | 1    | Version (`2`)                        |
| 1      | 1    | Encoding type                        |
| 2      | 2    | View factory                         |
| 4      | 2    | View type                            |
| 6      | 4    | Sender                               |
| 10     | 4    | Flow                                 |
| 14     | 8    | Sno                                  |
| 22     | 2    | Channel id                           |
| 24     | 2    | Channel name length (`-1` if absent) |
| 26     | *n*  | Channel name                         |

These offsets correspond to the `V1_MESSAGE_*_POS` and `V2_MESSAGE_*_POS` constants in `MessageMetadata`, and the fixed lengths to `FIXED_WIRE_LENGTH_V1` and `FIXED_WIRE_LENGTH_V2`.

{% hint style="info" %}
Read the version byte at offset 0 first and branch on it. Do not assume V2: which version a binding emits is configurable, and V1 is still what you get by default on some bindings. On the Solace binding, `sma_metadata_version=2` selects V2, see [Solace Binding](/talon/concepts-and-architecture/messaging-model/solace-binding).
{% endhint %}

## Sending Messages from External Applications

An application that does not run on Talon can still exchange messages with one. It has to do by hand what the binding would otherwise do: serialize the payload, build the metadata, and put both where the binding expects to find them.

The mechanics are per-binding, and each binding's page carries a worked example:

* [Solace Binding: Sending and Receiving from External Applications](/talon/concepts-and-architecture/messaging-model/solace-binding#sending-and-receiving-from-external-applications), using `x-sma-metadata` on a `BytesMessage`.
* [JMS Binding: Sending and Receiving from External Applications](/talon/concepts-and-architecture/messaging-model/jms-binding#sending-and-receiving-from-external-applications).

The field semantics those examples rely on are the ones documented above.

## Related Topics

* [Messaging Model](/talon/concepts-and-architecture/messaging-model) - channels, keys, and inbound channel resolution
* [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type)
* [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages)


# Microservice Architecture

The diagram above illustrates the Talon microservice architecture.

* Below the dashed line constitute the Talon runtime of a microservice.
* Above the dashed line are user developed artifacts and code

This section contains the following subsections:

* [Runtime Architecture](/talon/concepts-and-architecture/microservice-architecture/runtime-architecture) provides a high level description of the components that comprise the Talon microservice runtime.
* [Development Model](/talon/concepts-and-architecture/microservice-architecture/development-model) describes the the various artifacts a developers works with and the role that the Application Data Modeler (ADM) plays in the development process.
* [Configuration Model](/talon/concepts-and-architecture/microservice-architecture/configuration-model) provides a high level overview of the architecture of the Talon configuration subsystem and the artifacts used by a developer to populate the configuration repository - the repository from where the Talon runtime components source their configuration.


# Runtime Architecture

<figure><picture><source srcset="/files/68Kv7KkJ187qudbObPRh" media="(prefers-color-scheme: dark)"><img src="https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-58e4248b942da049b54f94e1f6a21b6261b014cf%2Fservice-arch-light.png?alt=media" alt=""></picture><figcaption></figcaption></figure>

The following is what constitutes the Talon microservice runtime:

### Atomic Event Processor (AEP)

The AEP engine is central to every Talon microservice, acting as an intermediary between the message bus, microservice logic, and data storage. It implements a cluster consensus model, allowing Talon microservices to be clustered for fault tolerance and high availability without data loss. The cluster consensus design ensures exactly- once message processing, simplifying the microservice programming model.

### Simple Messaging API (SMA)

Talon's Simple Messaging API (SMA) module offers a POJO-based messaging API that encapsulates necessary messaging primitives for the platform. This design enables seamless integration with any messaging provider through easily implemented bindings. SMA introduces the concepts of buses and channels, where applications send and receive messages as POJOs over bus channels, mapped via configuration to the underlying messaging provider destination.

### Object Data Store (ODS) <a href="#conceptsandarchitecture-operationaldatastore-ods" id="conceptsandarchitecture-operationaldatastore-ods"></a>

Talon's Object Data Store (ODS) offers a robust, clustered solution for storing POJO-based data objects with transactional integrity and durability. It facilitates rapid object retrieval and storage at memory speeds, while leveraging asynchronous, pipelined replication for consistent data views across cluster members. The AEP engine utilizes these features along with SMA's guaranteed message delivery to achieve exactly-once message processing semantics

Important ODS Concepts include:

* **In-Memory Access**: ODS implements the machinery to ensure that data and message objects managed by the store are always presented to store users in local memory i.e. put and gets always happen at memory speeds.
* **Replication**: The primary means of achieving persistence for an ODS Store is by pipelined, memory-memory replication of data objects bracketed by transactions to one or more hot backup peers. If a primary application instance fails the backup takes over with no loss of data.
* **Persistence**: An ODS Store also logs replicated message and data objects to binary transaction logs to ensure zero loss recovery from planned or unplanned downtime of the entire storage cluster.
* **Change Data Capture (CDC):** To allow asynchronous yet transactionally consistent syphoning of application message and data objects to back end or legacy systems, ODS supports the ability to perform **C**hange **D**ata **C**apture with a simple callback-based mechanism to push state changes.
* **Inter Cluster Replication (ICR): I**nter-**C**luster **R**eplication allows replication of the store's recovery stream to another data center over messaging, providing an asynchronous and transactional consistent disaster recovery mechanism.

### XVM

A Talon XVM is a runtime deployment container for Talon AEP Engines (microservices). It provides deployment management capabilities for the engines (microservices) that it contains. Key features include:

* Lifecycle management of engines and applications (message and event handlers).
* Alerting and Lifecycle Event emission.
* Collection and publication of system, platform and application level telemetry.
* Remote command and control of XVMs, engines and applications.

### Discovery

Talon uses a soft state distributed protocol for discovery of Talon runtime components. Although a generic capability, its primary use is the discovery of AEP engines for cluster formation and XVMs for telemetry connections.


# Development Model

<figure><picture><source srcset="/files/68Kv7KkJ187qudbObPRh" media="(prefers-color-scheme: dark)"><img src="https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-02d0b1d3c4588cecb9dca4f060802834dccf81b0%2Fservice-arch.png?alt=media" alt=""></picture><figcaption></figcaption></figure>

The following are the ingredients of a Talon application that a microservice developer works with:

* XML Models
  * A message model
    * One message model shared across multiple microservices
  * State models (optional)
    * One state model per microservice
* User code
  * The HA Policy Declarer
  * Lifecycle Methods
  * Microservice Initializers
  * Message Filters & Handlers
* Platform configuration

### Message & State Models

The **Application Data Modeler (ADM)** component of Talon employes an XML-based modeling language to define application messages and microservice stores. These XML models are inputs for the Talon code generator, which produces Plain Old Java Objects (POJOs) for the application's business logic. These objects are optimized for performance, enhancing developer productivity by managing object serialization, transport and persistence.

The following are sample message and state models

{% code title="Sample Message Model" fullWidth="false" %}

```xml
<?xml version="1.0"?>
<model xmlns="http://www.neeveresearch.com/schema/x-adml"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       namespace="com.example.helloapp.messages"
       defaultFactoryId="1">
    <factories>
      <factory name="HelloAppMessageFactory" id="1"/>
    </factories>
    <messages>
        <message name="HelloRequest" id="1">
            <field name="fromName" type="String" id = "1"/>
        </message>
        <message name="HelloReply" id="2">
            <field name="text" type="String" id = "1"/>
            <field name="count" type="Long" id = "2"/>
        </message>
    </messages>
</model>
```

{% endcode %}

{% code title="Sample State Model" fullWidth="false" %}

```xml
<?xml version="1.0"?>
<model xmlns="http://www.neeveresearch.com/schema/x-adml"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       namespace="com.example.helloapp.service.store"
       defaultFactoryId="2">
    <factories>
      <factory name="HelloServiceStoreFactory" id="1"/>
    </factories>
     
    <entities>
      <entity name="Store" id="100">
        <field name="counter" type="Long" id = "1"/>
      </entity>
    </entities>
</model>
```

{% endcode %}

See [**Modeling Message and State**](/talon/developing-applications/modeling-messages-and-state) for more information on modeling messages and state.

### User Code

The following is the user code for a basic Talon microservice. This illustrative microservice processes a `HelloRequest` message by updating a counter in its database and returns a `HelloReply` message with the updated counter value and an arbitrary string.

```java
@AppHAPolicy(value = StateReplication)
public class Main {
    private AepMessageSender sender;

    /**
     * Invoked by Talon runtime at startup to initialize the microservice 
     */
    @AppInitializer
    public void init() {
        System.out.println("Initialized");
    }
  
    /**
     * Invoked by the Talon microservice runtime to inject a message sender for use by the user code
     */
    @AppInjectionPoint
    public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    /**
     * Invoked by Talon microservice runtime just before it connects to the underlying messaging provider
     */
    @EventHandler
    public void onMessagingPrestart(final AepMessagingPrestartEvent event) {
        System.out.println("About to start messaging");
    }
  
    /**
     * Invoked by Talon microservice runtime to fetch the root of the microservice store
     */
    @AppStateFactoryAccessor
    public IAepApplicationStateFactory getStateFactory() {
        return new IAepApplicationStateFactory() {
            @Override
            final public Store createState(MessageView view) {
                return Store.create();
            }
        };
    }
      
    /**
     * Called by the Talon microservice runtime on receipt of a HelloRequest message.
     */
    @EventHandler
    public void onMessage(HelloRequest helloRequest, Store store) {
        // update store
        store.setCounter(store.getCounter() + 1);
        
        // send hello reply
        HelloReply helloReply = HelloReply.create();
        helloReply.setText("Hi There");
        helloReply.setCounter(store.getCounter());
        messageSender.sendMessage("hello-replies", helloReply);
    } 
}
```

The Talon runtime drives all user code within a Talon microservice. User-written code falls into the following two categories:

* Lifecycle methods
* Message handlers

#### Lifecycle Methods

The Talon runtime oversees the lifecycle of a Talon application by executing user-defined methods to handle various lifecycle functions:

* **Accessor Methods**\
  The Talon runtime invokes such methods to gather user-specific data necessary for its operation. The `getStateFactory()` method above is invoked to obtain the factory used to instantiate the root object of the microservice's store. This ensures Talon can instantiation the microservice store at the appropriate point in microservice lifecycle
* **Injection Methods**\
  The Talon runtime invokes such methods to supply the user code with handles to Talon runtime objects for use by the user code in its operation. The `setMessageSender()` method above us an example of such a method. Talon invokes this method, if implemented, to supply the user code with a an instance of the `AepMessageSender` object that the user code uses to send outbound messages.
* **Notification Methods**\
  The Talon runtime invokes such methods to notify the user of lifecycle and alert related events. The `init()` and the `onMessagingPrestart()` methods are examples of such methods. The `init()`method is invoked to notify the user code that the main microservice class - `HelloService` - was just loaded while `onMessagingPrestart()` is invoked to notify the user code that the Talon runtime is about to connect to the underlying configured messaging bus.

#### Message Handlers

Business logic within Talon microservices is driven by messages. This means all business logic is executed through message handlers, which are methods marked with the `@EventHandler` annotation and recognized by their method signature. The `onMessage()` method above is an example of a message handler.

Message Handlers are single-threaded and should generally be non-blocking. They consume messages, perform business logic, query and/or update state and send outbound messages. The Talon runtime ensures that message receipt, state updates, and outbound sends are atomic, reliable, and loss-free, even in case of failures.

See [**Authoring User Code**](/talon/developing-applications/authoring-user-code) for more information on writing event and message handlers.


# Configuration Model

<figure><picture><source srcset="/files/68Kv7KkJ187qudbObPRh" media="(prefers-color-scheme: dark)"><img src="https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-02d0b1d3c4588cecb9dca4f060802834dccf81b0%2Fservice-arch.png?alt=media" alt=""></picture><figcaption></figcaption></figure>

This page provides a conceptual overview of how Talon microservices are configured, including the Domain Descriptor Language (DDL) and global environment properties.

## Overview

Talon microservices don't interact directly with infrastructure components. Configuration separates business logic from operational concerns like messaging, storage, and high availability. Talon provides two complementary configuration mechanisms for configuring microservices:

1. **DDL (Domain Descriptor Language)**: XML-based configuration for components (buses, applications, XVMs)
2. **Global Environment**: Property-based configuration for runtime behavior (statistics, optimization, trace levels)

## Two Configuration Mechanisms

### DDL (Domain Descriptor Language)

DDL is an XML-based configuration schema that defines microservice components and their settings. It is primarily used to configure:

* Message buses and channels
* Applications (event handlers, storage, high availability)
* XVMs (execution containers)
* System details and metadata
* And more...

**Example DDL**:

```xml
<model>
  <!-- Global environment configuration -->
  <env>
    <nv>
      <msg.latency.stats>true</msg.latency.stats>
      <optimizefor>latency</optimizefor>
    </nv>
  </env>

  <systemDetails>
    <name>trading-system</name>
    <version>1.0.0</version>
  </systemDetails>

  <buses>
    <bus name="market-data" descriptor="solace://192.168.1.100:55555&vpn_name=trading">
      <channels>
        <channel name="orders" qos="Guaranteed" join="true">
          <key>order.new</key>
          <key>order.cancel</key>
        </channel>
      </channels>
    </bus>
  </buses>

  <apps>
    <app name="order-processor" mainClass="com.example.OrderProcessor">
      <messaging>
        <buses>
          <bus name="market-data">
            <channels>
              <channel name="orders" join="true"/>
            </channels>
          </bus>
        </buses>
      </messaging>

      <storage>
        <enabled>true</enabled>
        <haPolicy>EventSourcing</haPolicy>
      </storage>

      <captureTransactionLatencyStats>true</captureTransactionLatencyStats>
    </app>
  </apps>

  <xvms>
    <xvm name="trading-xvm">
      <apps>
        <app>order-processor</app>
      </apps>

      <heartbeats enabled="true" interval="5">
        <channels>
          <channel bus="market-data" name="stats"/>
        </channels>
      </heartbeats>

      <!-- XVM-specific environment configuration -->
      <env>
        <nv>
          <conservecpu>true</conservecpu>
        </nv>
      </env>
    </xvm>
  </xvms>
</model>
```

**Accessing DDL Configuration in User Code**:

User code accesses DDL configuration through **component descriptors** that are populated from the DDL:

```java
@AppInjectionPoint
public void setEngineDescriptor(AepEngineDescriptor descriptor) {
    // Descriptor loaded from DDL <app> configuration
    // Access settings configured in DDL
    boolean statsEnabled = descriptor.getCaptureTransactionLatencyStats();

    // Can also programmatically augment DDL configuration
    descriptor.setAdaptiveBatchingEnabled(true);
}

@AppInjectionPoint
public void setMessageBusDescriptor(MessageBusDescriptor descriptor) {
    // Descriptor loaded from DDL <bus> configuration
    String busName = descriptor.getName();
    // Access bus settings...
}
```

{% hint style="info" %}
**Talon Runtime Usage**: The Talon runtime uses the same mechanism internally - it creates component descriptors from the DDL to configure buses, stores, engines, and other components.
{% endhint %}

### Global Environment

The global environment provides property-based configuration for Talon runtime behavior. It is primarily used to configure:

* Statistics collection settings
* Optimization modes (latency vs throughput)
* Trace logging levels
* Discovery configuration
* Platform-wide defaults
* And more...

**Global Environment Sources** (in order of increasing precedence):

1. **System Properties** (lowest precedence)

   ```bash
   java -Dnv.msg.latency.stats=true -Dnv.optimizefor=latency ...
   ```
2. **App Property File**

   ```properties
   # app.properties
   nv.msg.latency.stats=true
   nv.optimizefor=latency
   nv.sma.trace=info
   ```

   Specify file via System property: `java -Dnv.app.propfile=app.properties ...`

   Or via environment variable: `export nv_app_propfile=app.properties`
3. **Environment Variables**

   ```bash
   export nv_msg_latency_stats=true
   export nv_optimizefor=latency
   ```
4. **DDL `<env>` Section** (highest precedence)

   ```xml
   <model>
     <env>
       <nv>
         <msg.latency.stats>true</msg.latency.stats>
         <optimizefor>latency</optimizefor>
         <sma.trace>info</sma.trace>
       </nv>
       <myapp>
         <maxOrderSize>1000000</maxOrderSize>
       </myapp>
     </env>
     <!-- ... rest of DDL ... -->
   </model>
   ```

{% hint style="info" %}
**Precedence**: Properties in the DDL `<env>` section override all other sources. System properties have the lowest precedence.
{% endhint %}

**Accessing Global Environment in User Code**:

The **preferred mechanism** is to use the `@Configured` annotation to inject properties from the DDL `<env>` section:

```java
@Configured(property = "myapp.maxOrderSize", defaultValue = "100000")
private int maxOrderSize;

@Configured(property = "myapp.tradingVenue", defaultValue = "NYSE")
private String tradingVenue;
```

**Alternatively**, use the `XRuntime` API for programmatic access:

```java
import com.neeve.ci.XRuntime;

// Get property value programmatically
String value = XRuntime.getValue("myapp.maxOrderSize", "100000");
int maxOrderSize = Integer.parseInt(value);
```

{% hint style="info" %}
**Talon Runtime Usage**: The Talon runtime uses `XRuntime` and `UtlEnv` APIs internally to access global configuration properties for statistics collection, optimization, trace levels, and other runtime behavior.
{% endhint %}

### DDL vs Global Environment

| Aspect                    | DDL                                                                     | Global Environment                                          |
| ------------------------- | ----------------------------------------------------------------------- | ----------------------------------------------------------- |
| **Format**                | XML elements and attributes                                             | Property key-value pairs                                    |
| **Configures**            | Components (buses, apps, xvms, channels)                                | Runtime behavior (stats, optimization, trace)               |
| **Accessed In User Code** | Via component descriptors                                               | Via @Configured annotation or XRuntime API                  |
| **Accessed By Platform**  | Via component descriptors                                               | Via XRuntime and UtlEnv APIs                                |
| **Example**               | `<captureTransactionLatencyStats>true</captureTransactionLatencyStats>` | `nv.msg.latency.stats=true`                                 |
| **Typical Use**           | Configure bus connections, app settings, XVM heartbeats                 | Enable stats, set optimization mode, configure trace levels |

### When to Use Each

**Use DDL** when:

* Configuring message bus connections and channels
* Configuring application messaging, storage, and HA settings
* Configuring XVM heartbeats and managed applications
* Setting component-specific parameters
* Configuration is specific to a component instance

**Use Global Environment** when:

* Enabling runtime statistics collection
* Setting optimization modes (latency vs throughput)
* Configuring trace logging levels
* Setting platform-wide defaults
* Configuration applies globally across the runtime

{% hint style="info" %}
**Note**: Some configuration settings can be specified using either mechanism. For example, trace levels for certain AEP engine loggers can be set via DDL or via global environment properties. When both mechanisms are available, **using DDL is recommended** as it keeps configuration centralized in the DDL file.
{% endhint %}

### Complete Configuration Example

Here's a complete example showing both mechanisms working together:

```xml
<model>
  <!-- GLOBAL ENVIRONMENT: Runtime configuration -->
  <env>
    <nv>
      <!-- Statistics configuration -->
      <msg.latency.stats>true</msg.latency.stats>
      <event.latency.stats>true</event.latency.stats>
      <stats.series.samplesize>10240</stats.series.samplesize>

      <!-- Optimization -->
      <optimizefor>latency</optimizefor>

      <!-- Trace logging -->
      <sma.trace>info</sma.trace>
      <aep.trace>fine</aep.trace>
    </nv>

    <!-- Application-specific properties -->
    <orderprocessor>
      <maxOrderSize>1000000</maxOrderSize>
      <tradingVenue>NYSE</tradingVenue>
    </orderprocessor>
  </env>

  <!-- DDL: Component configuration -->
  <systemDetails>
    <name>trading-system</name>
    <version>1.0.0</version>
  </systemDetails>

  <buses>
    <bus name="market-data" descriptor="solace://${SOLACE_HOST}:55555&vpn_name=${SOLACE_VPN}">
      <channels>
        <channel name="orders" qos="Guaranteed" join="true">
          <key>order.new</key>
          <key>order.cancel</key>
        </channel>
      </channels>
    </bus>
  </buses>

  <apps>
    <app name="order-processor" mainClass="com.example.OrderProcessor">
      <messaging>
        <buses>
          <bus name="market-data">
            <channels>
              <channel name="orders" join="true"/>
            </channels>
          </bus>
        </buses>
      </messaging>

      <storage>
        <enabled>true</enabled>
        <haPolicy>EventSourcing</haPolicy>
      </storage>

      <captureTransactionLatencyStats>true</captureTransactionLatencyStats>
    </app>
  </apps>
</model>
```

Accessing the configuration in user code:

```java
public class OrderProcessor extends EventHandler {
    // Inject from global environment
    @Configured(property = "orderprocessor.maxOrderSize", defaultValue = "100000")
    private int maxOrderSize;

    @Configured(property = "orderprocessor.tradingVenue", defaultValue = "NYSE")
    private String tradingVenue;

    // Access DDL configuration via descriptor
    @AppInjectionPoint
    public void setEngineDescriptor(AepEngineDescriptor descriptor) {
        // captureTransactionLatencyStats came from DDL
        boolean statsEnabled = descriptor.getCaptureTransactionLatencyStats();
    }

    @AppEventHandler
    public void onNewOrder(NewOrderMessage order) {
        // Use injected configuration
        if (order.getSize() > maxOrderSize) {
            // Reject order...
        }
    }
}
```

## Configuration Architecture

Now that we understand the two configuration mechanisms at a high level, let's dive into the architectural details of how Talon processes and stores configuration.

### Configuration Repository

Internally, Talon uses a **configuration repository** to store component configuration parsed from the DDL. The repository is an internal platform component that developers don't interact with directly.

**Key characteristics**:

* The repository stores configuration in a proprietary format keyed by **hierarchical names**
* Components are uniquely identified by hierarchical names comprised of their type and unique name
  * Example: An AEP engine named "order-processor" has the repository name `/aep/engines/order-processor`
* The Talon runtime uses these hierarchical names internally when loading descriptors from the repository
* The repository implements the [`com.neeve.config`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/config/package-summary.html) API (internal to Talon runtime)

{% hint style="info" %}
**Note**: Developers never directly interact with the configuration repository. Platform components and user code interact with higher-level descriptor objects, not the repository itself.
{% endhint %}

### Global Environment Storage

The global environment is stored in a map-like structure maintained by the Talon runtime. It is populated from multiple sources during initialization and provides a unified view of all runtime properties.

**Characteristics**:

* Assembled from System properties, app propfile, environment variables, and DDL `<env>` section
* DDL `<env>` properties have **highest precedence**
* XVM-specific `<env>` sections are merged with the global `<env>` section
* Accessed via `XRuntime.getValue()` and `UtlEnv` APIs

### Environment Sources and Precedence

The global environment is populated from multiple sources. When the same property is defined in multiple sources, the source with higher precedence wins.

**Sources in increasing precedence order** (lowest to highest):

1. **System Properties** (lowest precedence)
   * Java system properties (`-Dproperty=value`)
2. **App Propfile**
   * Properties file specified via `nv_app_propfile` or `-Dnv.app.propfile`
   * Overrides System properties
3. **Environment Variables**
   * OS-level environment variables
   * Override App propfile
4. **DDL `<env>` Section** (highest precedence)
   * Properties specified in the `<env>` element
   * Union of global `<env>` and XVM-specific `<env>` sections
   * Highest precedence (overrides all other sources)

**Example** showing precedence:

```bash
# System property (lowest precedence)
java -Dnv.optimizefor=throughput ...
```

```properties
# app.properties (overrides System property)
nv.optimizefor=latency
```

```xml
<!-- DDL (highest precedence - wins!) -->
<env>
  <nv>
    <optimizefor>none</optimizefor>
  </nv>
</env>
```

**Result**: `nv.optimizefor` will be `none` because DDL `<env>` has highest precedence.

{% hint style="warning" %}
**Important**: Some global platform properties CANNOT be set in the `<env>` section because they are needed before the DDL can be parsed. These must be specified using System properties, app propfile, or environment variables. See [Configuration Reference](/talon/reference/configuration) for the "Can Set in `<env>`?" indicator for each property.
{% endhint %}

### How DDL and Global Environment Interact

Both mechanisms work together during application initialization:

1. **Assemble Global Environment**: The Talon runtime collects properties from System properties, app propfile, environment variables, and merges them (with appropriate precedence)
2. **Parse DDL File**: The Talon runtime reads and parses the XML configuration file
3. **Merge DDL `<env>` Section**: Properties from the DDL `<env>` section are merged into the global environment (with highest precedence)
4. **Variable Substitution**: During DDL parsing, variable references like `${VARNAME::DEFAULT}` are substituted from the global environment
5. **Populate Repository**: Component configuration from DDL elements (`<buses>`, `<apps>`, `<xvms>`) is stored in the configuration repository
6. **Create Components**: The Talon runtime creates components by loading descriptors from the repository and using global environment for runtime configuration

**Example showing interaction**:

```xml
<model>
  <!-- Global environment provides substitution values -->
  <env>
    <solace.host>192.168.1.100</solace.host>
    <solace.vpn>trading</solace.vpn>
    <nv.msg.latency.stats>true</nv.msg.latency.stats>
  </env>

  <!-- DDL uses environment variables for substitution -->
  <buses>
    <bus name="market-data" descriptor="solace://${solace.host}:55555&vpn_name=${solace.vpn}">
      <!-- ... -->
    </bus>
  </buses>
</model>
```

In this example:

* `${solace.host}` is substituted with `192.168.1.100` from the global environment
* `${solace.vpn}` is substituted with `trading` from the global environment
* `nv.msg.latency.stats` configures runtime statistics collection
* The `<bus>` element populates the configuration repository for component creation

## Configuration Lifecycle

The configuration of Talon components is tied closely to the application lifecycle. Understanding this lifecycle helps clarify when and how configuration is applied.

### Configure Phase

When an application is launched, the first task is to populate both the configuration repository and global environment before any Talon runtime machinery is invoked. This ensures the runtime machinery picks up the correct configuration.

**Steps**:

1. Assemble the **global environment** from System properties, app propfile, environment variables, and DDL `<env>` section (with DDL `<env>` having highest precedence)
2. Parse the **DDL file** and populate the **configuration repository**
3. Use the environment for variable substitution during DDL parsing

### Run Phase

Once the configuration repository and environment are populated, the application can create and use platform components.

**Steps for creating a component**:

1. Create an instance of the component descriptor (e.g., [`AepEngineDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngineDescriptor.html))
2. Load the descriptor contents from the configuration repository using its `load()` method
3. The descriptor initializes itself using configuration from the repository (keyed by the component's hierarchical name)
4. Create the component using the loaded descriptor

{% hint style="info" %}
**Note**: The hierarchical naming and repository access happen internally. Developers work with descriptor objects, not the repository directly.
{% endhint %}

## Populating Configuration from DDL

How the DDL populates the configuration repository depends on whether Talon is used in an embedded or non-embedded manner.

### Embedded Use

Talon should be used in an **embedded manner** when the application's main entry point is in the application code.

#### XML Descriptor (RECOMMENDED)

Talon allows applications to describe configuration in XML form (DDL) and programmatically initialize the repository using the XML descriptor.

* **Schema**: Configuration is defined by the Domain Descriptor Language (DDL) schema, [`x-ddl.xsd`](https://build.neeveresearch.com/core/schema/LATEST/x-ddl.xsd)
* **Configuration Class**: Use [`VMConfigurer`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/config/VMConfigurer.html) to configure the Talon runtime from an XML descriptor
* **Recommendation**: This is the recommended mechanism if configuration can be stored in or converted to XML form

**Example**:

```java
import com.neeve.config.VMConfigurer;

public class MyApplication {
    public static void main(String[] args) throws Exception {
        // Initialize repository from DDL
        VMConfigurer.configure("config.xml");

        // Now create and use platform components
        // ...
    }
}
```

#### Component Descriptors (PROGRAMMATIC)

Each Talon component has a companion configuration descriptor for programmatic configuration. Descriptors implement setters/getters for configuration attributes and can save properties to the repository.

This mechanism is suited when configuration cannot be easily transformed to XML form.

**Component Descriptors**:

| Component            | Descriptor Class                                                                                                                                        | Description                                                           |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| SMA Bus              | [`MessageBusDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageBusDescriptor.html)                                   | Configures a message bus                                              |
| SMA Channel          | [`MessageChannelDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageChannelDescriptor.html)                           | Configures a bus channel                                              |
| ODS Store            | [`StoreDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/StoreDescriptor.html)                                             | Configures the store for an engine (providing HA)                     |
| ODS Store Replicator | [`StoreReplicatorDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/StoreReplicatorDescriptor.html)                         | Configures store replication to backup members                        |
| ODS Persister        | [`StorePersisterDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/StorePersisterDescriptor.html)                           | Configures store disk persistence                                     |
| ODS ICR              | [`StoreInterClusterReplicatorDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/StoreInterClusterReplicatorDescriptor.html) | Configures store inter-cluster replication (e.g., remote data center) |
| AEP Engine           | [`AepEngineDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngineDescriptor.html)                                     | Configures the application's AepEngine                                |
| XVM (Talon XVM)      | [`SrvConfigDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/config/SrvConfigDescriptor.html)                           | Configures an XVM                                                     |

**Example** (augmenting XML configuration programmatically):

```java
@AppInjectionPoint
public void setEngineDescriptor(AepEngineDescriptor descriptor) {
    // Descriptor already loaded from repository via XML
    // Augment with programmatic settings
    descriptor.setAdaptiveBatchingEnabled(true);
    descriptor.setAdaptiveBatchingCeiling(100);
}
```

{% hint style="info" %}
**Tip**: With the exception of SrvConfigDescriptor, applications can augment configuration seeded by the XML descriptor programmatically using component descriptors.
{% endhint %}

#### Configuration Script (INTERNAL)

Talon supports an internal scripting format for configuration. This format is not recommended for external use and is listed for informational purposes only.

### Non-Embedded Use

Talon is used in a **non-embedded manner** when the application's main entry point is in the Talon XVM.

In this mode:

* The configuration repository is initialized from external storage via a persistence plugin
* The recommended approach is to use deployment tools like Robin for configuration, deployment, and management
* Robin processes the same XML descriptor but materializes it from external storage rather than the filesystem

## Accessing Environment Properties in Application Code

The **preferred mechanism** for accessing user-defined properties from the global environment is to declare them in the DDL `<env>` section and inject them using the `@Configured` annotation:

```xml
<env>
  <myapp>
    <maxOrderSize>1000000</maxOrderSize>
    <tradingVenue>NYSE</tradingVenue>
  </myapp>
</env>
```

```java
@Configured(property = "myapp.maxOrderSize", defaultValue = "100000")
private int maxOrderSize;

@Configured(property = "myapp.tradingVenue", defaultValue = "NYSE")
private String tradingVenue;
```

See [Injecting Configuration](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) for complete details on using `@Configured`.

**Alternative programmatic access** is available via the [`XRuntime`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html) API:

```java
import com.neeve.ci.XRuntime;

// Get a property value programmatically
String value = XRuntime.getValue("myapp.maxOrderSize", "100000");
int maxOrderSize = Integer.parseInt(value);
```

{% hint style="info" %}
**Note**: The [`UtlEnv`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/util/UtlEnv.html) API is also available but should only be used when directed by Neeve support.
{% endhint %}

## DDL Features

The DDL provides several powerful features for managing configuration:

### Variable Substitution

DDL supports variable substitution using the syntax `${VARNAME::DEFAULT}` where:

* `VARNAME` is the property name to substitute
* `DEFAULT` is an optional default value if the property is not found

Variables are substituted from the global environment (System properties, app propfile, environment variables, DDL `<env>`).

**Example**:

```xml
<env>
  <solace.host>192.168.1.100</solace.host>
  <solace.port>55555</solace.port>
</env>

<buses>
  <bus name="market-data" descriptor="solace://${solace.host}:${solace.port::55555}">
    <!-- ... -->
  </bus>
</buses>
```

If `solace.host` is not defined, the substitution fails. If `solace.port` is not defined, it defaults to `55555`.

**Substitution Sources** (in precedence order):

1. System properties
2. App propfile
3. Environment variables
4. DDL `<env>` section (highest precedence)

**Special Characters in Property Names**:

* `.` (period) is used as a hierarchical separator in XML
* `-` (dash) and `_` (underscore) are allowed in property names

**Example with hierarchical properties**:

```xml
<env>
  <nv>
    <msg>
      <latency>
        <stats>true</stats>
      </latency>
    </msg>
  </nv>
</env>
```

This creates the property `nv.msg.latency.stats=true`.

### DDL Overrides

Any DDL attribute or element value can be set or overridden at runtime using specially-named system properties. DDL overrides serve two purposes:

1. **Set values** - Provide configuration values for elements not present in the DDL XML
2. **Override values** - Change configuration values that are present in the DDL XML

This allows configuration to be externalized without modifying the DDL file, which is particularly useful for environment-specific configuration, runtime tuning, and CI/CD deployments where the same DDL is used across environments.

#### Override Property Naming Pattern

DDL override properties follow a hierarchical naming pattern that mirrors the XML structure:

```
[prefix].[section].[instance-key].[path-to-element]
```

**Components:**

* **Prefix**: `x.` (default, configurable via `nv.ddl.override.prefix` system property)
* **Section**: Top-level DDL section (`busProviders`, `buses`, `apps`, `xvms`)
* **Instance Key**: The `name` attribute value of the instance (e.g., bus name, app name, XVM name)
* **Path**: Dot-separated path following the XML hierarchy to the target element or attribute

#### Basic Examples

**Example 1: Override an existing value**

Override a bus descriptor defined in DDL:

```xml
<buses>
  <bus name="market-data" descriptor="solace://localhost:55555"/>
</buses>
```

```bash
# Override the descriptor at runtime
-Dx.buses.bus.market-data.descriptor=solace://prod-host:55555
```

**Example 2: Set a value not in DDL**

Set storage configuration without it being in the DDL:

```xml
<apps>
  <app name="order-processor" mainClass="com.example.OrderProcessor">
    <!-- No storage configuration in DDL -->
  </app>
</apps>
```

```bash
# Set storage configuration via override (no DDL needed)
-Dx.apps.order-processor.storage.enabled=true
-Dx.apps.order-processor.storage.persistence.enabled=true
-Dx.apps.order-processor.storage.persistence.flushOnCommit=false
```

The above creates the storage configuration as if it were present in the DDL.

**Example 3: Override nested settings:**

Change an existing nested value in the DDL:

```xml
<apps>
  <app name="order-processor">
    <storage>
      <persistence enabled="true">
        <flushOnCommit>false</flushOnCommit>
      </persistence>
    </storage>
  </app>
</apps>
```

```bash
# Override the flushOnCommit value at runtime
-Dx.apps.order-processor.storage.persistence.flushOnCommit=true
```

**Example 4: Set XVM heartbeat settings:**

Set heartbeat configuration that doesn't exist in DDL:

```xml
<xvms>
  <xvm name="trading-xvm">
    <!-- No heartbeat configuration in DDL -->
  </xvm>
</xvms>
```

```bash
# Set heartbeat configuration via overrides
-Dx.xvms.trading-xvm.heartbeats.enabled=true
-Dx.xvms.trading-xvm.heartbeats.interval=10
```

**Example 5: Override existing XVM heartbeat settings:**

Change an existing heartbeat interval value:

```xml
<xvms>
  <xvm name="trading-xvm">
    <heartbeats enabled="true" interval="5"/>
  </xvm>
</xvms>
```

```bash
# Override the interval value at runtime
-Dx.xvms.trading-xvm.heartbeats.interval=10
```

#### Special Cases

**1. Key Attributes (Not Overridable)**

Attributes that serve as keys (typically `name` attributes) cannot be overridden because they uniquely identify instances:

```xml
<bus name="market-data">  <!-- name cannot be overridden -->
<app name="my-app">       <!-- name cannot be overridden -->
```

**2. Environment Properties**

Properties in the `<env>` section use the `x.env.` prefix for DDL overrides but are accessed directly (without prefix) in code:

```xml
<env>
  <nv>
    <msg.latency.stats>true</msg.latency.stats>
  </nv>
</env>
```

```bash
# Set or override via DDL override (uses x.env. prefix)
-Dx.env.nv.msg.latency.stats=false

# Or set directly as system property (no prefix, becomes env property)
-Dnv.msg.latency.stats=false
```

The `x.env.*` pattern allows you to set environment properties that aren't in the DDL `<env>` section, or override ones that are.

**3. Template Configuration**

Template properties use a special pattern with the `templates` keyword:

```xml
<buses>
  <templates>
    <template name="solace-template">
      <provider>solace</provider>
    </template>
  </templates>
</buses>
```

```bash
# Override template property
-Dx.buses.templates.solace-template.provider=jms
```

**4. Nested Keyed Elements**

When child elements have their own `name` attribute, include it in the path:

```xml
<bus name="market-data">
  <channels>
    <channel name="orders">
      <qos>Guaranteed</qos>
    </channel>
  </channels>
</bus>
```

```bash
# Note: uses channel name directly (not "channel.orders")
-Dx.buses.market-data.orders.qos=BestEffort
```

#### Setting Override Properties

Override properties can be set via multiple mechanisms:

**1. System Properties (command line):**

```bash
java -Dx.apps.order-processor.storage.persistence.flushOnCommit=true \
     -Dx.buses.market-data.orders.qos=BestEffort \
     -jar myapp.jar
```

**2. Environment Variables:**

```bash
export x_apps_order_processor_storage_persistence_flushOnCommit=true
export x_buses_market_data_orders_qos=BestEffort
```

**3. App Properties File:**

```properties
# app.properties
x.apps.order-processor.storage.persistence.flushOnCommit=true
x.buses.market-data.orders.qos=BestEffort
```

Specify the properties file via: `-Dnv.app.propfile=app.properties`

#### Override Precedence

When both DDL and override properties are specified:

1. DDL Override properties (highest precedence)
2. DDL XML values (lowest precedence)

This allows runtime values to override static configuration.

#### Pattern Construction Examples

| XML Element Path                                                | Override Property                          |
| --------------------------------------------------------------- | ------------------------------------------ |
| `<busProviders><provider name="custom">`                        | `x.busProviders.custom.*`                  |
| `<buses><bus name="mkt-data" descriptor="...">`                 | `x.buses.bus.mkt-data.descriptor`          |
| `<buses><bus name="mkt-data"><channels><channel name="orders">` | `x.buses.mkt-data.orders.*`                |
| `<apps><app name="myapp" mainClass="...">`                      | `x.apps.myapp.mainClass`                   |
| `<apps><app name="myapp"><storage><persistence enabled="true">` | `x.apps.myapp.storage.persistence.enabled` |
| `<xvms><xvm name="myxvm"><heartbeats interval="5">`             | `x.xvms.myxvm.heartbeats.interval`         |

For complete override property documentation for all DDL elements, see the [Configuration Reference](/talon/reference/configuration).

### DDL Templates

**Since 3.8**: DDL templates reduce configuration repetition by allowing you to define reusable configuration blocks.

**Example**:

```xml
<buses>
  <templates>
    <template name="solace-template">
      <provider>solace</provider>
      <properties>
        <vpn_name>trading</vpn_name>
        <username>app</username>
        <password>secret</password>
      </properties>
    </template>
  </templates>

  <bus name="market-data" template="solace-template">
    <address>192.168.1.9</address>
    <port>55555</port>
  </bus>

  <bus name="reference-data" template="solace-template">
    <address>192.168.1.10</address>
    <port>55555</port>
  </bus>
</buses>
```

Templates are applied first, then bus-specific settings override template values.

### DDL Profiles

**Since 3.8**: DDL profiles enable environment-specific configuration within a single DDL file.

**Example**:

```xml
<model>
  <!-- Default configuration -->
  <buses>
    <bus name="market-data" descriptor="solace://localhost:55555">
      <!-- ... -->
    </bus>
  </buses>

  <!-- Production profile -->
  <profile name="production">
    <buses>
      <bus name="market-data" descriptor="solace://prod-solace.example.com:55555">
        <!-- ... -->
      </bus>
    </buses>
  </profile>

  <!-- Development profile -->
  <profile name="development">
    <buses>
      <bus name="market-data" descriptor="loopback://market-data">
        <!-- ... -->
      </bus>
    </buses>
  </profile>
</model>
```

**Activating a profile**:

```bash
# Via system property
java -Dnv.ddl.profile=production ...

# Via environment variable
export nv_ddl_profile=production
```

When a profile is activated, its configuration overrides the default configuration.

### DDL Processing Order

The DDL is processed in the following order:

1. **Parse DDL file**: Read and validate XML structure
2. **Assemble global environment**: Merge System properties, app propfile, environment variables, DDL `<env>` (with precedence)
3. **Apply active profile**: If a profile is specified, merge profile configuration
4. **Perform variable substitution**: Replace `${VARNAME::DEFAULT}` references from environment
5. **Apply DDL overrides**: Apply command-line `x.*` overrides
6. **Populate repository**: Store component configuration in repository

## Troubleshooting Configuration

### Enabling DDL Trace

To troubleshoot configuration issues, enable DDL trace logging:

```bash
# Must use System property (cannot use DDL <env> section)
java -Dnv.ddl.trace=true ...
```

This outputs detailed information about:

* Variable substitution
* Profile activation
* DDL parsing and validation
* Property precedence resolution

DDL trace output goes to the `nv.ddl` logger at `INFO` level.

### Common Issues

**Variable substitution fails**:

* Check that the property is defined in one of the environment sources
* Verify property name spelling (case-sensitive)
* Enable DDL trace to see substitution details

**Property has wrong value**:

* Check precedence order (DDL `<env>` has highest precedence)
* Enable DDL trace to see which source provided the value
* Verify profile activation if using profiles

**Property cannot be set in `<env>` section**:

* Some properties (like `nv.ddl.trace`) must be set via System property because they're needed before DDL parsing
* See [Configuration Reference](/talon/reference/configuration) for the "Can Set in `<env>`?" indicator

## Related Topics

* [Injecting Configuration](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) - Using @Configured annotation in application code
* [Configuration Reference](/talon/reference/configuration) - Complete DDL elements and global properties reference
* [Configuring the Runtime](/talon/developing-applications/configuring-the-runtime) - Feature-specific configuration guides

## Next Steps

1. Understand the difference between DDL (component configuration) and Global Environment (runtime properties)
2. Review the [Configuration Reference](/talon/reference/configuration) for complete DDL syntax and global properties
3. Learn how to use DDL templates and profiles for environment portability
4. Use [Injecting Configuration](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) to access configuration in your application code
5. Enable DDL trace logging to troubleshoot configuration issues


# Microservice Operation

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Lifecycle

This page documents the lifecycle of a Talon microservice, describing the order in which microservices are started and shutdown, and the events emitted that allow microservices to hook into this lifecycle.

## Overview

The lifecycle of a Talon microservice is a composite of the following:

* The lifecycle of the microservice as driven by the Talon XVM
* The lifecycle of the microservice's AEP engine
* The lifecycle of the microservice's AEP engine's message bus bindings
* The lifecycle of the microservice's AEP engine's store

This page describes each of these lifecycles and their constituent flows:

**Engine Lifecycle:**

* Create Engine
* Start Engine
* Activate Engine
* Stop Engine

**Store Lifecycle:**

* Open Store Binding
* Close Store Binding

## Microservice Lifecycle

A Talon microservice's lifecycle starts when the microservice main class is loaded by the Talon XVM and ends when the XVM invokes the method on the microservice's main class annotated with `@AppFinalizer`. The following diagram depicts the microservice flow from the Talon XVM's standpoint:

![Microservice Lifecycle](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-5f9b03befe6b63049e59f340c7878c84d2ea1f8e%2F18941065.png?alt=media)

### Load Phase

#### Load Microservice Main Class

The first step executed by the Talon XVM is to load the microservice main class.

#### Discover Lifecycle Methods

After loading the microservice main class, the Talon XVM introspects the loaded class for lifecycle-related methods. Lifecycle methods are identified via annotations and are optional. Methods not found by the XVM are skipped during the microservice lifecycle.

### Open Phase

#### Inject Application Loader

The Talon XVM injects the application loader into the microservice using the loader injection method on the main microservice class. The application loader provides facilities to access additional objects, such as the microservice and XVM configuration descriptors.

```java
@com.neeve.server.app.annotations.AppInjectionPoint
public void setLoader(final com.neeve.server.app.SrvAppLoader loader) {...}
```

#### Get HA Policy

The Talon XVM uses the `@AppHAPolicy` annotation on the main microservice class to query the microservice's HA Policy.

```java
@com.neeve.server.app.annotations.AppHAPolicy(value=EventSourcing)
```

#### Prepare Engine Descriptor

The Talon XVM instantiates the descriptor used to configure the microservice's AEP Engine. The descriptor is loaded from the X configuration repository if present; otherwise, a fresh default descriptor is instantiated. The HA policy, if obtained in the last step, is set in the engine descriptor.

#### Inject Engine Descriptor

The Talon XVM injects the engine descriptor into the microservice for any modifications the microservice would like to make on the descriptor.

```java
@com.neeve.server.app.annotations.AppInjectionPoint
public void setEngineDescriptor(final com.neeve.aep.AepEngineDescriptor descriptor) {...}
```

#### Configuration Injection

{% hint style="info" %}
Since 3.2
{% endhint %}

After the engine descriptor is prepared, the Talon XVM injects configuration values from the [`XRuntime`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html) environment into objects provided by the microservice.

**Discover Config Injected Objects:**

```java
@com.neeve.server.app.annotations.AppConfiguredAccessor
public void addConfiguredObjects(Set<Object> objects) {...}
```

**Config Injection:**

After retrieving the set of config injected objects, the Talon XVM introspects the objects and injects configuration as described in the configuration documentation.

#### Get Command Handler Containers

The Talon XVM fetches the set of microservice objects that contain command handlers:

```java
@com.neeve.server.app.annotations.AppCommandHandlerContainerAccessor
public void addCommandHandlerContainerObjects(Set<Object> containers) {...}
```

**Discover Command Handlers:**

Command handler methods are annotated with `@AppCommandHandler`:

```java
@com.neeve.server.app.annotations.AppCommandHandler(command="printhelloworld")
public String helloWorld(String command, String[] args) {
   System.out.println("Hello World!");
}
```

{% hint style="info" %}
Command handler container objects can also contain non-command handler methods. Those methods will be ignored by the XVM command handler parser machinery.
{% endhint %}

#### Get AppStat Containers

The Talon XVM discovers additional objects that contain `@AppStats`:

```java
// Note that the method name is unimportant.
@com.neeve.server.app.annotations.AppStatContainersAccessor
public void addAppStatContainerObjects(Set<Object> containers) {...}
```

**Discover AppStats:**

AppStats can be exposed by fields or methods on the microservice's main class or in AppStatContainers by annotating with `@AppStat`. See [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) for additional details.

#### Get Event Handler Containers

The Talon XVM fetches the set of microservice objects that contain event handlers:

```java
@com.neeve.server.app.annotations.AppEventHandlerContainersAccessor
public void addEventHandlerContainerObjects(Set<Object> containers) {...}
```

**Discover Event Handlers:**

Event handler methods are annotated with `@EventHandler`. Event handlers are single-argument methods that contain an event or a message type as their argument:

```java
@com.neeve.aep.annotations.EventHandler
public void onEngineActivated(final com.neeve.aep.event.AepEngineActiveEvent event) {...}
```

```java
@com.neeve.aep.annotations.EventHandler
public void onOrder(final NewOrderMessage message) {...}
```

{% hint style="info" %}
Event handler container objects can contain non-event handler methods. Those methods will be ignored by the XVM and AEP engine event handler parser machinery.
{% endhint %}

#### Get Default Event Handler

The XVM fetches the microservice's default event handler via the method annotated with `@AppEventHandlerAccessor`:

```java
@com.neeve.server.app.annotations.AppEventHandlerAccessor
public IEventHandler getDefaultEventHandler() {...}
```

Depending on configuration, the default event handler is used to either:

* Dispatch events not handled by any other event handler (`DefaultHandlerDispatchPolicy=DispatchIfNoAnnotatedHandlers` - default)
* Always dispatch events regardless of whether dispatched to other annotation-based event handlers (`DefaultHandlerDispatchPolicy=DispatchAlways`)

#### Get Application State Factory

The final step before creating the microservice's engine is to fetch the microservice's state factory (for State Replicated microservices):

```java
@com.neeve.server.app.annotations.AppStateFactoryAccessor
public IAepApplicationStateFactory getStateFactory() {...}
```

#### Create Engine

At this point, the XVM has all the information needed to create the microservice's AEP engine. See [Engine Lifecycle](#engine-lifecycle) below for details on the engine creation flow.

#### Inject Engine

After creating the engine, the XVM injects the engine into the microservice's main class:

```java
@com.neeve.server.app.annotations.AppInjectionPoint
public void setEngine(final com.neeve.aep.AepEngine engine) {...}
```

#### Initialize Application

The final step in the open phase is to initialize the microservice:

```java
@com.neeve.server.app.annotations.AppInitializer
public void initialize() {...}
```

### Start Phase

#### Start Engine

The only act performed in the start phase is to start the engine. The act of starting the engine determines whether the started engine is the primary or a backup in the microservice's cluster. If primary, messages start flowing to the microservice. If backup, messages and/or state are replicated in real time from the primary to the backup to keep the backup's state in sync with the primary. If the primary fails, the backup elected as primary opens its messaging machinery and messages start flowing to the microservice for processing. See [Engine Lifecycle](#engine-lifecycle) below for details on the engine start flow.

### Run Phase

#### Event Driven Applications

Talon microservices are message/event driven. For such microservices, nothing additional needs to be done to enter the run phase. Once the engine has been started, messages and/or events are dispatched to the microservice, driving its operation.

#### Synchronous Applications

For microservices that are not event driven (i.e., "sender" only type microservices that synchronously drive their own operation), the XVM provides the facility to implement the "main" method. The XVM identifies the method in the microservice's main class annotated with `@AppMain` to be the microservice's main method. If such a method is discovered, the XVM spins up a separate thread that invokes the microservice main method:

```java
@com.neeve.server.app.annotations.AppMain
public void main() {...}
```

### Stop Phase

#### Stop Engine

The only act performed in the stop phase is to stop the engine. See [Engine Lifecycle](#engine-lifecycle) below for details on the engine stop flow.

### Close Phase

#### Finalize Application

The last step executed by the Talon XVM in the microservice lifecycle is to invoke the microservice's finalize method:

```java
@com.neeve.server.app.annotations.AppFinalizer
public void finalize() {...}
```

## Engine Lifecycle

The following depicts the overall lifecycle of an AEP engine:

![Engine Lifecycle](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-bb2cfcd3d3e3014d9d683a99c2789421956e16bf%2F18941079.png?alt=media)

### Create Engine

The following depicts the creation flow of an AEP engine:

![Engine Create](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-f1a6030a873f765f09c13cb73b75666d25bc1fd3%2F18941071.png?alt=media)

During engine creation, the engine:

1. Initializes internal components
2. Sets up message processing infrastructure
3. Prepares for messaging and store bindings
4. Configures event dispatching

### Start Engine

The following depicts the start flow of an AEP engine:

![Engine Start](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-d30c362764221f28eeacbd5df122b6bdaf31ab2f%2F18941077.png?alt=media)

During engine start, the engine:

1. Participates in cluster consensus to determine role (primary or backup)
2. Initializes based on determined role
3. Prepares to activate if elected primary

### Activate Engine

The following depicts the activation flow of an AEP engine:

![Engine Activate](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-c6b987e71fda2f59ba80ac171f893c939fd11d76%2F18941075.png?alt=media)

During engine activation:

1. The engine opens messaging connections
2. Joins configured channels
3. Opens store bindings if configured
4. Begins processing messages

This phase marks the transition to the engine becoming fully operational and processing messages.

### Stop Engine

The following depicts the stop flow of an AEP engine:

![Engine Stop](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-4eb15a390a8bd8689593d03ca799d3bfec0409be%2F18941073.png?alt=media)

During engine stop:

1. Messaging is stopped and connections are closed
2. Store bindings are closed
3. Internal resources are cleaned up

## Store Lifecycle

### Open Store Binding

The following depicts the open flow of an ODS store:

![Store Open](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-2e7ea67824d0d2fb73af059d27b3b640fd415315%2F18941067.png?alt=media)

When a store binding opens:

1. Connection to the persistent store is established
2. The binding joins the store cluster
3. State initialization occurs (from persistent store or from primary)
4. The binding becomes operational

### Close Store Binding

The following depicts the closure flow of an ODS store:

![Store Close](https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-43f7c2f6b42daf3e5bb26f41816f72821f2188f6%2F18941069.png?alt=media)

When a store binding closes:

1. Any pending transactions are completed or rolled back
2. The binding leaves the store cluster
3. Connection to the persistent store is closed
4. Resources are released

## Event Reference

### Engine Events

| Event Type                                                                                                                                  | Description                                                                                                                                                                                                                                                                                                                                                                           |
| ------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`AepEngineCreatedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineCreatedEvent.html)               | Dispatched when the AEP engine has been successfully created.                                                                                                                                                                                                                                                                                                                         |
| [`AepMessagingPrestartEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepMessagingPrestartEvent.html)       | Dispatched before the engine attempts to establish message bus bindings and join any configured channels.                                                                                                                                                                                                                                                                             |
| [`AepMessagingStartedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepMessagingStartedEvent.html)         | Dispatched after the engine has attempted to establish message bus bindings and join any configured channels. Dispatched after corresponding channel up events.                                                                                                                                                                                                                       |
| [`AepMessagingStartFailedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepMessagingStartFailedEvent.html) | Dispatched when the engine fails to start its messaging machinery. Dispatched after the engine has attempted to establish bindings. For bindings that were successfully established, corresponding binding and channel up events would have been dispatched before this event.                                                                                                        |
| [`AepEngineStartedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineStartedEvent.html)               | Dispatched when the AEP engine has been successfully started.                                                                                                                                                                                                                                                                                                                         |
| [`AepEngineActiveEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineActiveEvent.html)                 | Dispatched when the engine has been elected as primary and has successfully started its messaging machinery. Dispatched after the engine has successfully established message bus bindings and joined any configured channels. Corresponding binding and channel up events are dispatched before this event.                                                                          |
| [`AepFlowCreatedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepFlowCreatedEvent.html)                   | Dispatched when the AEP engine creates a new AepFlow. AepFlows are created when a message is processed either during steady state on a primary or backup instance or during recovery for a flow that does not yet exist in the engine.                                                                                                                                                |
| [`AepStateCreatedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepStateCreatedEvent.html)                 | Dispatched when an instance of the microservice store has been created. This event is only dispatched on a backup AEP engine instance.                                                                                                                                                                                                                                                |
| [`AepMessagingFailedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepMessagingFailedEvent.html)           | Dispatched when the engine has shut down its messaging due to a failure. Dispatched after the engine has shut down all established message bus bindings. Whether and how an engine decides to shut down messaging is determined by the [`MessageBusBindingFailPolicy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.MessageBusBindingFailPolicy.html). |
| [`AepEngineStoppedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineStoppedEvent.html)               | Dispatched when the AEP engine has been stopped.                                                                                                                                                                                                                                                                                                                                      |

### Messaging Events

| Event Type                                                                                                                                      | Description                                                                                                                                                                                                                             |
| ----------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`AepBusBindingCreatedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingCreatedEvent.html)           | Dispatched when the engine has created a binding to a message bus. Dispatched before the binding is opened or started.                                                                                                                  |
| [`AepBusBindingCreateFailedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingCreateFailedEvent.html) | Dispatched when the engine encounters a failure when trying to create a bus binding.                                                                                                                                                    |
| [`AepBusBindingOpeningEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingOpeningEvent.html)           | Dispatched just before the engine starts opening a binding to a message bus. Followed by a binding open or binding open fail event.                                                                                                     |
| [`AepBusBindingOpenedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingOpenedEvent.html)             | Dispatched when the engine has successfully opened a bus binding.                                                                                                                                                                       |
| [`AepBusBindingOpenFailedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingOpenFailedEvent.html)     | Dispatched when the engine encounters a failure when trying to open a bus binding.                                                                                                                                                      |
| [`AepChannelUpEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepChannelUpEvent.html)                           | Dispatched when the engine has successfully connected to the bus containing a channel configured to be of interest to the microservice. Guaranteed to precede any messages arriving through that channel.                               |
| [`AepBusBindingUpEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingUpEvent.html)                     | Dispatched when the engine has successfully established a binding to a message bus. Dispatched after the channel up events for the established binding and guaranteed to precede any messages arriving through the established binding. |
| [`AepChannelDownEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepChannelDownEvent.html)                       | Dispatched when the engine has successfully disconnected from the bus containing a channel configured to be of interest to the microservice. Guaranteed to succeed any messages arriving through that channel.                          |
| [`AepBusBindingDownEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingDownEvent.html)                 | Dispatched when an operational bus binding fails.                                                                                                                                                                                       |
| [`AepBusBindingDestroyedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepBusBindingDestroyedEvent.html)       | Dispatched when the engine has destroyed a binding to a message bus.                                                                                                                                                                    |

### Store Events

| Event Type                                                                                                                              | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| --------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`IStoreBindingFailedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBindingFailedEvent.html)           | Dispatched to indicate that a store binding has 'failed'. A binding 'failure' is a binding closure triggered implicitly by the binding on the occurrence of certain events that the binding deems fatal enough that it cannot continue operations. On a failure, a binding performs all closure operations, transitions to the failed state, and then dispatches this event. Although permissible, it is not necessary to close a binding on a failure (since it is already implicitly closed). Note that, if you choose to close the binding subsequent to a failure, the close should not be invoked from within the binding event handler. Doing so may cause a deadlock.                                                                                                                                   |
| [`IStoreBindingRoleChangedEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBindingRoleChangedEvent.html) | Dispatched to notify that the role of the member represented by a store binding is changing. Dispatched before the binding's new role takes effect.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| [`IStoreMemberUpEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreMemberUpEvent.html)                     | Notifies that a new member has joined an ODS store.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| [`IStoreMemberDownEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreMemberDownEvent.html)                 | Notifies that a member has left an ODS store.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| [`IStoreMemberInitCompleteEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreMemberInitCompleteEvent.html) | <p>Dispatched in the following situations:<br>1. On primary members after initialization from the persister is complete.<br>2. On primary members when a new member joining the store has been successfully initialized. Follows the <code>IStoreMemberUpEvent</code> for the new member.<br>3. On standalone receiver members after initialization from the persister is complete.<br>4. On backup members after initialization from the primary is complete. Follows the <code>IStoreMemberUpEvent</code> for the primary member.<br><br>Listen for the <code>IStoreBindingRoleChangedEvent</code> to keep track of role and appropriately interpret this event, or compare the member reported in this event with the member returned by <code>IStoreBinding.getMember()</code> to interpret the event.</p> |

## See Also

* [Initialization](/talon/concepts-and-architecture/microservice-operation/cluster-initialization) - Microservice initialization
* [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) - Steady state message processing
* [Cluster Join](/talon/concepts-and-architecture/microservice-operation/cluster-join) - The cluster join process
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How cluster consensus is achieved
* [Cluster Failover](/talon/concepts-and-architecture/microservice-operation/cluster-failover) - How cluster failover occurs
* [Implementing Lifecycle Methods](/talon/developing-applications/authoring-user-code/lifecycle/implementing-lifecycle-methods) - How to implement lifecycle hooks in your microservices


# Initialization

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Cluster Join

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Message Processing

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Cluster Consensus

From a microservice perspective Talon's message processing flow is not dissimilar to many other event or message processing architectures: the microservice exposes message handlers to the platform which in turn passes inbound messages to it for processing. In the act of processing the inbound event, the microservice will make changes to its state and send some outbound events. The key difference between the Talon architecture and traditional architectures is that microservice state is stored in memory and resiliency is provided not by synchronous persistence to disk or a data grid, but instead by streaming state changes to a backup's memory in an asynchronous, pipelined fashion. The combination of in-memory state and asynchronous 'persistence' allows Talon to operate at extreme performance levels without sacrificing on reliability.

Key aspects of any streaming application platform that are of critical concern for stream oriented applications are:

* **Exactly once processing** of inbound and outbound messages.
* **Atomicity** between **state updates** and the **messaging stream**: A trading application that thinks it has sent a request to buy 100,000 shares of IBM must be able to rely on that being what is actually sent out. This is particularly important in application or machine failure scenarios. If the application fails after sending out the request to buy 100,000 IBM and the application were to recover and reprocess the event this time buying 200,000 shares it could turn into a costly business problem.

In traditional application architectures the problems of state/messaging atomicity and exactly once processing are often solved using distributed transactions. For example, in J2EE state changes would be committed to a databases and messaging sent over JMS with XA transactions used to coordinate commit on both. Such schemes involve a lot of overhead and kill performance. The application flow in Talon provides the same level of reliability without the synchronous overhead of distributed transaction coordination.

The following diagram depicts the message processing flow in a Talon microservice that ensures state consensus between the microservice cluster members:

Elaborating on the diagram above, Talon ensures the same level of reliability as traditional architectures as follows:

1. The AEP Engine receives an inbound message
2. It starts a transaction and dispatches the message to the microservice
3. Microservice updates state (monitored by the engine)
4. Microservice sends outbound messages (through the engine)
5. The engine holds onto the outbound messages until the message handler completes and control reaches the engine. At that point, the engine starts the process of establishing consensus with the other cluster members. The first step step is to replicate the state changes and outbound messages as an atomic unit to the backup instances via memory to memory replication.
6. Once the state and outbound messages (processing "effects") are stabilized on a backup, the backup notifies the primary via an asynchronous stability acknowledgement
7. The primary then releases the outbound messages and sent downstream.
8. The engine receives acknowledgements of all outbound messages sent i.e. the downstream receiver - message broker or end receivers - have stabilized the messages.
9. The engine then acknowledges the inbound message upstream indicating to the upstream sender that it has stabilized the inbound message.

{% hint style="info" %}
**Pipelining**

The AEP engine executes the above in a non-blocking pipelined manner of execution. What this means is that the engine does not block on any of the operations. At any point in time, there are several transactions in the pipeline being executed concurrently - or, more accurately, in a time sliced manner
{% endhint %}

**Key Takeaways**

* The stabilization of state changes and outbound effects to the backup before sending outbound messages ensures that in the event of process or machine failure that the backup has the same conception of state as the former primary communicated externally (e.g. I have an outstanding request to buy 100,000 shares of IBM).
* The acknowledgement of the inbound message after replication to the peer ensures that in the event of failure a duplicate can be detected to prevent duplicate dispatch to the application if the message is re-transmitted.
* The entire flow above is pipelined. The application can begin processing the next inbound message before receiving stability from the backup. Nowhere in the flow is there a need to block or perform synchronous operations.
* State and outbound messages can be journaled to a transaction log on disk, but it is not fsync'd and is not a primary recovery mechanism. Disk based journaling is used more for operational tasks, but can also serve as a backup recovery mechanism.
* And best of all ... the above is done transparently to the application which need only concern itself with writing the business logic in message handlers.


# Cluster Failover

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Consensus Models

**Work in Progress**: This page is currently being developed. Content will be added in a future update.


# Transactions

## Overview

As described in [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) and [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus), the AEP engine uses transactions to ensure state consensus in a service cluster. While a microservice processes messages, the AEP engine prepares and atomically replicates transactions in a manner that ensures the primary instance's store is identically consistent with the store on each of the backup instances in the cluster.

## Transaction Elements

What constitutes a transaction is determined by the service consensus model. The followng lists the contents of a transaction:

* **State Replicated Microservices**
  * Inbound message metadata
  * Service store change log
  * Outbound messages sent outbound as part of the inbound message processing
* **Event Sourced Microservices**
  * Inbound message metadata
  * Inbound messages

By default, each inbound message starts and terminates a transaction. However, the AEP engine can bundles multiple inbound messages into a single transaction and commit the entire batch as a single atomic unit. This is called *adaptive batching.*

## The Transaction Pipeline

When an inbound message enters the engine's event dispatch loop, the message is attached to the current transaction. As the engine routes the message through its handlers, it incrementally updates the [elements](#transaction-elements) of the transaction. On return from the message handler, the engine decides whether to commit the transaction or not. If it does, it

* Marks the transaction as complete
* Creates a new current transaction
* Dispatches the completed transaction for commitment i.e. replication and persistence across the cluster to achieve cluster consensus.

Transaction commitment occurs in a **pipelined** manner: while a transaction is being replicated across the cluster, the engine continues to process and commit new inbound messages under subsequent transactions. A transaction in the process of being committed is called an **inflight** transaction and the set of all such transactions is called the **transaction pipeline**.

## Transaction Commit Legs

Committing a transaction is performed in three stages, referred to as **legs**:

| Leg       | What it does                                                                                 |
| --------- | -------------------------------------------------------------------------------------------- |
| **Leg 1** | Store commit submission - hands the transaction to the store for replication and persistence |
| **Leg 2** | Send commit submission - submits the transaction's outbound messages to their bus bindings   |
| **Leg 3** | Commit completion - completes the transaction once every send commit has been acknowledged   |

Leg 1 and leg 3 are always executed by the engine thread. Leg 2 is the exception: when the runtime is optimized for latency, the engine executes it on the store thread the moment the store commit completes, rather than dispatching it back through the engine's multiplexer. Skipping that dispatch removes a measurable amount of latency from the commit path, which is why it is the default.

This means that under a latency optimized engine the commit is executed by two threads rather than one. The engine accounts for this internally: leg 3 will not complete a transaction while leg 2 is still working on it. In practice leg 2 finishes long before the send commits it submitted are acknowledged, so leg 3 does not wait.

The behaviour is controlled by two settings, both described in the [Configuration Reference](/talon/reference/configuration):

* **`leg2InStoreThread`** (default `true`) - set to `false` to execute leg 2 on the engine thread as well, making the commit single threaded. This has no effect unless the runtime is optimized for latency, and it costs latency, so it is intended as a diagnostic rather than a tuning option.
* **`leg2CompletionTimeout`** (default 10 seconds, in microseconds) - an upper bound on how long leg 3 will wait for leg 2. It exists so that an engine can never be held up indefinitely; if it expires the engine logs a severe message and completes the transaction anyway.

{% hint style="info" %}
Leg 2's duration is reported by the engine's transaction statistics, so the cost of this stage is visible without any special configuration.
{% endhint %}

## Adaptive Batching

By default, an AEP engine processes each inbound message in a single transaction. It is possible to configure the engine to batch up the processing of several inbound messages into a single transaction. This feature us called **adaptive batching**.

Adaptive batching can significantly improve throughput. However, this is generally at the cost of increased latency of outbound messages since the outbound messages for the first message processed in a transaction won't be sent until the last message in the transaction has been processed and the transaction dispatched for commit.

The batching behavior is adaptive in nature because the engine commits a transaction automatically when either a configured adaptive batch ceiling is reached or the engine detects there are no more messages immediately available to process. In other words, if there are messages arriving significantly fast that they can be added to the current transaction with no additional weight, then the batch size of the transaction will grow to the configured batch ceiling. However, if there is a slight lull in the inbound traffic pattern that would cause the engine to have to wait for the next message to fill the batch, then the engine does not wait and will immediately close the batch.


# Threading Model

Understanding Talon's threading architecture is essential to understanding how the platform achieves its extreme performance characteristics. This page explains the design principles, architectural decisions, and concepts that underpin Talon's threading model.

## Overview

Talon's threading model is built on two fundamental architectural principles that work together to achieve extreme performance levels. First, write access to the microservice store is single-threaded, eliminating the costly overhead of managing concurrent access to shared data. Second, work flows through a pipeline where critical processing steps can be offloaded to dedicated threads that hand work forward without blocking, keeping the main business logic thread focused on executing handlers.

These principles address the fundamental challenge in high-performance computing: when multiple threads contend for the same piece of data or resource, scalability suffers. The costs aren't just in locks and synchronization primitives—even with all state in main memory, multiple threads operating on the same data face significant costs at the processor cache level. Talon's architecture eliminates these costs entirely for microservice state.

## The Single Writer Principle

The [single writer principle](https://mechanical-sympathy.blogspot.com/2011/09/single-writer-principle.html) posits that when trying to build a highly scalable system, the single biggest limitation on scalability is having multiple writers contend for any item of data or resource. This principle motivates architectural patterns like the actor model and microservices, and it's fundamental to Talon's design.

Talon's microservice architecture makes all state private to each microservice, reducing write contention by partitioning data. By bringing all microservice data into memory, Talon further reduces the cost of updating data by keeping it as close to the business logic operating on it as possible. But even with all state in main memory, there are significant costs when multiple threads operate on the same piece of data—processor caches must be synchronized between cores, cache lines are invalidated, and memory has to be reloaded from higher levels of cache or main memory.

Every Talon microservice is backed by an AEP Engine with a single input multiplexer thread—the **dispatcher thread**—that consumes events and messages coming in from message buses and dispatches them to handler code. This thread serves as the single writer for the microservice's state. Handler code executes on this same thread, so all state modifications happen serially without any synchronization primitives. Application developers don't need to concern themselves with locks, mutexes, atomic operations, or thread-safe data structures.

As with most architectures, horizontal scalability can still be achieved by partitioning state across multiple microservice instances, whether in the same JVM, on the same machine, or across multiple machines. But the single writer architecture reduces the need for sharding by eliminating the hardware inefficiency of managing inter-thread contention. A single microservice instance can process far more transactions per second when it's not wasting processor and memory resources on synchronization.

## Detached Threads and Pipelining

While the microservice programming model is single-threaded, it's desirable to keep the dispatcher thread busy performing application logic rather than spending cycles on infrastructural concerns. The platform provides the ability to do much of the non-functional heavy lifting—like replication, persistence, and message I/O—in background threads that are "detached" from the business logic thread.

Work that can be offloaded to detached threads includes:

* **Store replication** - Sending state updates to backup instances (detached store sender)
* **Store replication dispatch** - Deserializing received replication traffic (detached store dispatcher)
* **Persistence** - Writing recovery logs to disk (detached persister)
* **Inter-cluster replication** - Sending state across clusters (detached ICR sender)
* **Message logging** - Audit logging of inbound/outbound messages (detached message loggers)
* **Bus I/O** - Serializing and sending outbound messages (detached bus sender)

This creates a processing pipeline. The dispatcher thread executes a handler, which modifies state and prepares outbound messages. Instead of blocking to replicate that state or serialize and send those messages, the dispatcher immediately hands that work forward to a detached thread and begins processing the next transaction. The detached threads work in parallel, each focused on their specific task—one thread replicates to backups, another persists to disk, another sends messages to the bus. The dispatcher just keeps executing handlers.

For optimal latency and throughput, these detached threads can be affinitized to particular CPU cores, reducing the performance impact of thread context switching—a concept we'll explore more in the next section.

## Disruptors: Inter-Thread Communication

Effective pipelining between threads requires optimal inter-thread communication. Talon uses [LMAX Disruptors](https://github.com/LMAX-Exchange/disruptor)—a high-performance inter-thread messaging library—to pass data between critical threads in the processing pipeline.

Disruptors implement a ring buffer with sophisticated wait strategies. When the dispatcher thread has work to hand forward—say, state updates that need to be replicated—it writes those updates into a ring buffer. The detached replication thread reads from that same buffer. The ring buffer is sized as a power of 2, typically 1024 entries, large enough to absorb spikes in traffic without blocking the offering thread but small enough to keep active data within CPU caches.

The wait strategy determines how a thread waiting for work behaves. For ultra-low latency, threads can busy spin—continuously checking for new work without yielding to the operating system. This keeps the CPU's instruction pipeline hot and avoids context switch jitter, but it requires dedicating a full CPU core to that thread. For better CPU utilization with slightly higher latency, threads can yield to the OS or even block, allowing other threads to use that core. The platform provides controls to configure these trade-offs.

## Thread Affinitization and NUMA

Modern server hardware adds another layer of complexity that Talon's threading model addresses. Contemporary CPUs typically have multiple cores—often 10 to 20 physical cores per socket. Servers often have multiple CPU sockets. And many CPUs support hyper-threading, where each physical core appears as two logical CPUs to the operating system.

This creates a **NUMA** (Non-Uniform Memory Access) architecture. Each CPU socket has its own bank of RAM—a NUMA node. A thread running on socket 0 can access memory on socket 0's NUMA node quickly, but accessing memory on socket 1's node requires going across the inter-socket link, which is significantly slower. For an in-memory computing platform like Talon, where the primary storage mechanism is memory, this matters enormously from a Von Neumann Bottleneck perspective.

**Thread affinitization**—pinning threads to specific CPU cores—addresses these challenges. When you pin the dispatcher thread to a specific core and pin all the detached threads to cores on the same socket, and you ensure the process's memory is allocated on that socket's NUMA node, you achieve several benefits:

* **Cache locality**: The dispatcher and its detached threads share the same L3 cache, minimizing the time to pass work between them
* **Memory locality**: All threads access memory on the local NUMA node, minimizing memory access latency
* **Eliminates context switching**: The OS won't move pinned threads to different cores, so processor caches stay hot

**Hyper-threading** presents a trade-off. With hyper-threading enabled, two logical CPUs share the same physical core's CPU caches, meaning each logical process has less cache space available, resulting in higher memory access time. For Talon applications optimized for latency, it's generally best to disable hyper-threading when possible to maximize the cache available to each thread.

## Goals of Affinitization

With the above concepts in mind, optimal performance for a Talon microservice is achieved when:

* All critical threads are affinitized to the same processor socket, sharing the same L3 cache
* Process memory is affinitized to that same socket's NUMA node, avoiding remote NUMA access
* Critical threads are pinned to their own CPU cores and set to busy spin, avoiding context switches
* Hyper-threading is disabled, preventing threads from being scheduled onto the same physical core as a busy-spinning thread

This level of tuning isn't necessary for all deployments—the benefits are most pronounced in ultra-low-latency applications where every microsecond matters. But understanding these concepts helps explain why Talon can achieve such extreme performance characteristics when properly configured.

## Configuration

Threading behavior is configured through DDL and system properties. See [Configuring Threading](/talon/developing-applications/configuring-the-runtime/threading) for detailed configuration guidance including:

* Disruptor configuration (queue depth, wait strategies)
* Thread affinitization (basic and advanced approaches)
* NUMA topology optimization
* Per-thread affinity masks

## Programming Implications

The single-threaded model profoundly simplifies application code. Handler code runs on the dispatcher thread, and since that's the only thread that modifies state, developers don't need thread synchronization. There's no need for locks, mutexes, volatile fields, or concurrent collections. State can be accessed directly without any thread-safety concerns.

This does impose one requirement: handler code must be deterministic and non-blocking. Blocking the dispatcher thread blocks the entire microservice—no other handlers can execute until the current one completes. This means avoiding blocking I/O, long-running computations, or calls to external services that might delay. The programming model trades away the complexity of concurrent programming for the discipline of writing fast, focused handlers.

See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) for detailed coding guidelines.

## Related Topics

* [Configuring Threading](/talon/developing-applications/configuring-the-runtime/threading) - Configure disruptors, affinitization, and NUMA
* [Runtime Architecture](/talon/concepts-and-architecture/microservice-architecture/runtime-architecture) - How threading fits into the overall runtime
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Operational threads and their roles

## Next Steps

1. Review [Configuring Threading](/talon/developing-applications/configuring-the-runtime/threading) to understand configuration options
2. Understand programming implications in [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals)
3. Learn about the [Runtime Architecture](/talon/concepts-and-architecture/microservice-architecture/runtime-architecture) to see how threads interact with other components


# Discovery Model

Understanding how Talon microservices discover each other to form clusters and establish operational connections.

## Overview

Talon provides built-in discovery mechanisms that simplify configuration and reduce coupling between components. Discovery enables:

* **Application Cluster Establishment**: Instances of the same application automatically discover each other to form an HA cluster based on the application name
* **XVM Discovery**: Allows administrative and monitoring tools to discover running XVMs and their applications

## Discovery Caches and Advertisements

A discovery provider provides the ability to load a discovery cache. A discovery cache allows its creator to advertise entities it owns by periodically broadcasting Entity State Advertisements (ESAs) to remote caches. Applications can register discovery event handlers with the cache to receive updates about newly added entities on remote caches or entities removed either explicitly by the remote cache or by virtue of stale entities "aging out" when ESAs are not received for an entity after some time.

### Entity State Advertisements

Advertisements use an internal binary protocol and communicate the following information for each entity. A discovery cache will periodically broadcast ESAs to prevent remote caches from declaring them dead.

| Field                          | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Entity Owner**               | The unique id of the discovery cache that advertised the entity                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| **Entity Type**                | <p>The type of entity. Talon uses:<br>- "Application" for Talon applications<br>- "OdsStoreMember" for Talon ODS stores<br>- "Server" for XVMs</p>                                                                                                                                                                                                                                                                                                                                                                                                                |
| **Entity Name**                | The unique name of the entity being advertised                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| **Entity Instance**            | The unique instance identifier for the entity. For example, an ODS Store Member would have the same name in each XVM for the same application but would have a uniquely identifiable instance id                                                                                                                                                                                                                                                                                                                                                                  |
| **Entity Host**                | The host from which the entity was advertised                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| **Entity Address Descriptors** | Each entity added to the cache includes a list of address descriptors that contain connection information                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| **Entity Age**                 | The entity's current age. See Entity Max Age for information on the role played by an entity's age in its lifecycle management                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| **Entity Max Age**             | <p>Each entity instance is associated with a maximum age and, at any point in time, a current age. A cache member 'expires' (removes from its local cache) a discovered entity when its current age reaches its maximum age. To prevent entity expiry, member owners periodically broadcast ESA (Entity State Advertisement) packets. The receipt of an ESA by non-owner causes the entity's age to be reset.<br><br>An entity's maximum age is also used (in conjunction with other configurable parameters) by owner members to determine when to send ESAs</p> |

### Discovery Cache Lifecycle

Discovery caches manage entity lifecycle through a combination of periodic advertisements and age-based expiration:

1. **Entity Advertisement**: When an entity is added to a cache, the cache begins periodically broadcasting ESAs for that entity
2. **Age Management**: Remote caches increment the age of discovered entities over time
3. **Age Reset**: When an ESA is received, the entity's age is reset to zero
4. **Expiration**: If no ESA is received before the entity reaches its maximum age, the entity is removed from remote caches
5. **Event Notification**: Applications can register handlers to be notified when entities are added or removed from the cache

## Discovery Providers

Discovery providers implement different mechanisms for transmitting ESAs between caches. Talon includes several built-in providers to support different deployment scenarios.

### Multicast

The multicast provider uses IP multicast for broadcasting ESAs. This is the default discovery provider.

**Characteristics:**

* Simple configuration with no infrastructure dependencies
* Well-suited for development and testing
* Limited to local network segment
* No central point of failure

**Default Configuration:**

* Default address: `mcast://224.0.1.200:4090`

**Considerations:**

* Must run over IPv4 stack (use `-Djava.net.preferIPv4Stack=true` on IPv6-preferring systems)
* Requires multicast-capable network
* On hosts with multiple network interfaces, you may need to specify which interface to use via the `localIfAddr` property
* On Mac OS X Yosemite and later, the virtual awdl0 interface can interfere with multicast discovery

### SMA (Messaging-Based)

The SMA discovery provider uses topic broadcasts over any broker-based SMA messaging provider for advertising ESAs.

**Characteristics:**

* Works across network boundaries
* Leverages existing messaging infrastructure
* Supports distributed deployments
* Can be configured to use dedicated messaging infrastructure separate from application traffic

**Configuration:**

* Uses `discoveryChannel` property to specify the channel/topic name (default: `_XEDP_`)
* Supports all SMA bus bindings (Solace, JMS, ActiveMQ, Kafka, etc.)
* Bus binding-specific properties can be passed through the discovery descriptor

{% hint style="info" %}
**Tip**: When using SMA-based discovery, consider using a different messaging server than the one used for application traffic, especially if monitoring tools rely on XVM discovery. A failure in the messaging fabric would otherwise leave applications undiscoverable.
{% endhint %}

### Local

The local discovery provider is a simple provider for finding entities within the same process.

**Characteristics:**

* Lowest overhead
* In-process only
* No network communication
* Useful for unit testing and development

**Use Cases:**

* Unit tests launching multiple applications in the same JVM
* Development environments with collocated services
* Testing scenarios requiring isolated discovery

## Discovery Usage Patterns

### Default Discovery

By default, Talon uses a single global discovery cache for all discovery needs within a JVM. Applications, XVMs, and ODS Stores all advertise via this default cache, which uses multicast by default.

### Separate Discovery Domains

In some scenarios, it's advantageous to use separate discovery providers for different purposes:

* **XVM Discovery**: Used by administrative and monitoring tools to find XVMs
* **Cluster Discovery**: Used by application instances to find peers and form HA clusters

Using separate discovery domains allows you to:

* Isolate administrative traffic from application cluster formation
* Use different network infrastructure for different discovery purposes
* Apply different discovery policies (timeout, max age) to different entity types

## Design Rationale

Discovery in Talon is designed around several key principles:

**Decoupling**: Applications and tools don't need to know specific network addresses. They discover peers dynamically, allowing deployments to change without configuration updates.

**Simplicity**: The default multicast configuration requires no infrastructure setup, making development and testing straightforward.

**Flexibility**: Multiple provider options and the ability to configure separate discovery domains support diverse deployment requirements from single-developer environments to large distributed systems.

**Resilience**: Age-based expiration with periodic advertisements provides automatic cleanup of failed instances while tolerating temporary network issues through configurable ESA loss tolerance.

## Configuration

Discovery is configured through DDL and environment properties. See:

* [Discovery Configuration](/talon/developing-applications/configuring-the-runtime/discovery) - Configuring discovery providers and descriptors
* [Operating Model](/talon/concepts-and-architecture/operating-model) - How discovery integrates with administration and monitoring

## Related Topics

* [Cluster Join](/talon/concepts-and-architecture/microservice-operation/cluster-join) - How microservices use discovery to join clusters
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Discovery's role in administration and monitoring
* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Using SMA-based discovery for administration

## Next Steps

1. Understand how discovery is used for [cluster formation](/talon/concepts-and-architecture/microservice-operation/cluster-join)
2. Configure discovery for your environment in [Discovery Configuration](/talon/developing-applications/configuring-the-runtime/discovery)
3. Use the [Discovery Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/discovery-tool) to troubleshoot discovery issues


# Operating Model

This page describes Talon's operating model—the architecture and mechanisms for administering, monitoring, and troubleshooting running applications.

## Overview

Operating Talon applications in production requires three complementary capabilities working in concert. Administration provides the control plane for managing running microservices, allowing operators to issue commands, query state, and manage lifecycle operations. Monitoring observes runtime behavior by collecting and delivering telemetry that reveals how the system is performing. Analysis and troubleshooting capabilities diagnose issues by examining logs and historical data when problems occur.

These three capabilities are deeply integrated through a common infrastructure. Discovery enables operational tools to locate running XVMs. Messaging channels carry both administrative commands and monitoring data. Transaction logs capture detailed history for post-mortem analysis. Together, they form a cohesive operating environment that scales from development through production.

## Administration

### Administrative Architecture

Talon recognizes that different operational contexts demand different administrative approaches, and provides two distinct administration modes to accommodate these varying needs.

In direct administration mode, XVMs advertise an admin acceptor through the discovery system. Admin clients discover these XVMs and establish direct TCP connections to them. This approach is particularly well-suited for development and diagnostic scenarios where low latency and direct access are valuable. The direct connection provides immediate, responsive interaction with individual XVMs without requiring additional infrastructure.

Admin over SMA (Shared Memory Appliance) takes a messaging-based approach instead. XVMs emit administrative messages over configured messaging channels, and admin clients subscribe to these channels through the message bus. This mode excels in production environments where you need to manage many distributed XVMs. It supports passive monitoring—observers can watch administrative traffic without requiring direct connections to XVMs. This architecture scales naturally as you add more XVMs, since all communication flows through the existing messaging infrastructure.

### Command and Control

Administration in Talon follows a command-and-control pattern where operators issue commands to running XVMs and receive responses. The system provides built-in XVM commands for common lifecycle and diagnostic operations like thread dumps, statistics queries, and configuration changes. Microservice developers can extend this by defining application-specific commands using `@AppCommandHandler` annotations, allowing custom operational procedures to be exposed through the same administrative interface.

Commands flow through request channels to their target XVMs, which process them and return responses via response channels. Discovery plays a crucial role here—commands target specific XVMs by name, and the discovery provider resolves these names to actual XVM locations. This decoupling means administrative tools don't need to know where XVMs are running; they simply address commands to the appropriate name and let discovery handle the routing.

### Design Rationale

The dual administration modes reflect different operational priorities. TCP mode provides low-overhead access ideal for interactive debugging and development, where latency matters and you're working with a small number of local XVMs. SMA mode scales to production deployments with hundreds of XVMs distributed across data centers, where establishing direct TCP connections would be impractical and where you want multiple monitoring tools observing the same administrative traffic.

Discovery decouples administrative clients from XVM locations, allowing XVMs to move, restart, or scale without reconfiguring administrative tools. The channel-based model in SMA mode means adding more monitoring tools or XVMs doesn't require connection management—everything flows through the established messaging infrastructure.

## Monitoring

### Monitoring Architecture

Monitoring in Talon is built around a simple but powerful concept: each XVM runs a background stats collection thread that periodically gathers metrics and emits them as heartbeats. This thread wakes up at regular intervals—typically every one to five seconds, though this is configurable—and walks through a hierarchy of statistics collectors. It starts at the XVM level, collecting JVM and system metrics, then descends into each engine to gather message processing statistics, and finally reaches into the application layer for custom metrics. As it collects, it also performs higher-level computations like calculating rates, averages, and percentiles from raw counters.

The result of this collection cycle is a heartbeat—a periodic message that serves two purposes. First, it's a "proof of life" signal showing that the XVM is running and healthy. Second, it carries a snapshot of all collected statistics, timestamped and tagged with the XVM's identity. These heartbeats form the foundation of Talon's monitoring capability.

Heartbeats can be delivered through three different mechanisms, each suited to different operational contexts. Trace output writes human-readable statistics to log files, which is convenient during development and diagnostics when you want to quickly scan metrics in a text editor. However, this approach creates garbage and isn't suitable for latency-sensitive production environments.

Binary transaction logs provide a zero-garbage alternative. Statistics are written in compact binary format to sequential log files, where they can be stored efficiently and analyzed offline using dedicated tools. This approach works well in production when you need to maintain ultra-low latency—writing to binary logs adds minimal overhead, and you can perform analysis later without impacting the running system.

The third mechanism, SMA channels, emits heartbeats over the messaging infrastructure where they can be consumed by remote monitoring applications. This enables real-time dashboards and alerts and scales naturally to distributed deployments. Monitoring tools simply subscribe to the heartbeat channels and receive statistics as they're emitted, without needing direct connections to XVMs.

### Statistics Hierarchy

The statistics collected at each heartbeat cycle form a natural hierarchy that mirrors Talon's architecture. At the top level, XVM statistics capture JVM health metrics like heap usage, garbage collection activity, and thread counts, along with system-level metrics such as CPU usage and load average. These process health indicators tell you whether the XVM itself is healthy.

Descending into the engine level, you find metrics that reveal how messages are being processed. Message throughput and rates show how much work is flowing through the system. Transaction counts and latencies indicate how long it takes to process work. Queue depths and backlog metrics reveal whether the system is keeping up with load. Consensus performance statistics show how well the cluster is coordinating. Store replication and persistence metrics track data durability operations.

At the application level, microservice developers define custom statistics using `@AppStat` annotations. These application-specific metrics might track business operations—orders processed, trades executed, inventories updated—or operational characteristics like cache hit rates or queue sizes. The ability to define custom statistics lets developers expose exactly the metrics that matter for their specific application.

### Latency Collection

Latency statistics deserve special attention because they're more expensive to collect than simple counters or gauges. Tracking latencies requires capturing timestamps at multiple points in message processing pipelines and computing differences, which adds overhead. Talon provides granular control over which latencies to collect, allowing operators to balance visibility against performance impact.

Message latencies track individual messages through their lifecycle—ingestion from the bus, queuing in the engine, processing by handlers, and transmission back out. Transaction latencies break down the transaction pipeline into stages, measuring time spent in handler code, persisting to the store, and committing outbound sends. Store latencies focus specifically on persistence and replication timing. For deeper investigation, you can enable per-message-type latencies, which provide detailed breakdowns for each message type, though this carries higher overhead. The most detailed option, per-transaction stats, captures a complete trace of every transaction, but this level of visibility comes with very high overhead suitable only for diagnostic scenarios.

Operators configure these latency collection options based on their needs. Development environments might enable everything to understand system behavior. Production environments typically enable basic latencies and selectively enable detailed collection when investigating specific issues.

### Design Rationale

Periodic collection strikes a balance between overhead and visibility—you get regular snapshots of system behavior without the cost of continuous instrumentation. The multiple delivery mechanisms reflect real operational needs: trace output for quick debugging, binary logs for production monitoring without garbage, and SMA channels for real-time dashboards. The hierarchical organization of statistics provides appropriate granularity at each level—you don't need application details to diagnose a JVM problem, but you do need them to understand business logic issues. Making expensive statistics opt-in lets operators tune the overhead/visibility trade-off for their specific situation.

## Analysis & Troubleshooting

### Trace Logging

When monitoring shows that something is wrong, trace logging helps you understand what's happening inside your microservices. Talon's trace logging system provides configurable runtime diagnostics through a three-layer architecture. At the bottom, tracers are objects embedded throughout Talon's code that can emit diagnostic messages. These tracers don't write directly to output; instead, they send their messages to loggers—named entities organized in a hierarchical namespace like `nv.aep`, `nv.ods`, and `nv.sma`. Each logger can be configured with its own trace level, allowing fine-grained control over what gets logged. Finally, handlers receive trace output from loggers and route it to destinations like stdout, stderr, files, network sockets, or memory buffers.

This architecture provides tremendous flexibility. Handlers can be daisy-chained to route trace to multiple destinations simultaneously. You can dynamically adjust trace levels at runtime without restarting microservices. The system integrates with both Talon's native logging and standard frameworks like SLF4J, letting you work with familiar tools.

One particularly clever feature is the memory handler, which buffers trace output in memory but only writes it out when triggered by a severe error. This captures the context leading up to problems—you get to see what was happening in the moments before a failure occurred—while minimizing trace overhead during normal operation. It's like having a flight recorder that's always running but only preserves data when something goes wrong.

### Transaction Logs

While trace logging shows you what code is executing, transaction logs provide a complete record of the data flowing through your system. These logs are the foundation for deep analysis and troubleshooting.

Talon uses several types of transaction logs, each serving a specific purpose. The recovery transaction log is fundamental to Talon's fault tolerance. In Event Sourcing mode, it records inbound messages so they can be replayed after a failure. In State Replication mode, it captures state tree updates—the PUTs, UPDATEs, and REMOVEs that modify your microservice's state. When a microservice recovers from a failure, it reads this log to restore itself to the correct state.

Beyond recovery, you can enable additional logs for audit and diagnostic purposes. An inbound message log records every message your microservice receives, regardless of whether it's needed for recovery. This creates a complete audit trail of incoming requests. Similarly, an outbound message log captures every message your microservice sends, enabling you to trace message paths through complex multi-hop flows. For the deepest level of diagnostics, per-transaction stats logs record detailed statistics for every transaction—complete latency breakdowns and all metrics—though this comes with high overhead and is typically reserved for diagnostic sessions.

All these logs use a compact binary format that enables zero-garbage writes, which is crucial for maintaining low latency in production. The format supports fast sequential access and indexing, making offline analysis efficient even with large log files.

### Query and Analysis Tools

Transaction logs would have limited value if you could only read them sequentially, but Talon provides powerful tools that turn these logs into queryable databases. The Transaction Log Tool provides both interactive browsing and XPQL—a SQL-like query language specifically designed for transaction logs.

The XPQL query engine maps transaction log entries to a relational table model. Each log becomes a table, each entry becomes a row, and you get columns for entry metadata plus typed columns for every message and entity type in your schema. You can build indexes on any field, enabling efficient queries even in logs containing millions of entries. This turns transaction logs into a powerful analytical database—you can run complex queries to find patterns, track specific messages through your system, or analyze behavior over time.

For statistics analysis, the Stats Dump Tool reads binary heartbeat logs and outputs human-readable statistics. This enables offline analysis without any impact on running systems. You can apply date range filters to focus on specific time periods, making it easy to investigate historical incidents or analyze performance trends.

### Design Rationale

The binary log format enables rich post-mortem analysis without impacting production performance—writes are zero-garbage and fast, while reads happen offline. Multiple log types serve different purposes, so you can enable exactly the logging you need without paying for capabilities you don't use. The query capability transforms logs from sequential records into a queryable database, dramatically expanding what you can learn from them. Having all analysis tools work offline means you can investigate issues as deeply as needed without affecting running systems.

## Integration: How It All Works Together

### Discovery as the Foundation

Discovery is the glue that binds all of Talon's operational capabilities together. XVMs advertise themselves through the configured discovery provider, making their presence known to the operational infrastructure. Both admin clients and monitoring tools use discovery to find XVMs—they don't need hardcoded addresses or complex configuration. The choice of discovery provider (multicast, UDP, SMA-based) determines your operational topology, and changing discovery providers changes how your operational tools connect without requiring changes to the tools themselves.

### Admin Channels Carry Monitoring Data

When you enable Admin over SMA, something elegant happens: the same messaging infrastructure that carries administrative commands also carries monitoring data. The xvm-heartbeat channel delivers statistics for monitoring. The xvm-trace channel carries trace output. The xvm-event channel broadcasts lifecycle events and alerts. The xvm-request and xvm-response channels handle administrative commands. This unified approach means you're not building and maintaining separate networks for administration versus monitoring—one messaging infrastructure serves both purposes.

### Transaction Logs Enable Analysis

Real-time monitoring through heartbeats and SMA channels shows you what's happening right now, but transaction logs capture history. While your monitoring dashboard displays current throughput and latency, transaction logs are quietly recording every message and transaction to disk. Later, when you need to understand what happened during an incident, offline query tools let you analyze past behavior without any impact on the running system. You can treat these logs like a database, running complex queries to understand what occurred.

The recovery transaction log serves double duty in this model. Its primary purpose is enabling failure recovery—when a microservice crashes and restarts, it reads this log to restore state. But this same log also enables replay for testing, message path analysis across microservice boundaries, and tracking how state evolved over time. What you write for fault tolerance also becomes a powerful analytical resource.

### Stats Flow Example

Consider the lifecycle of a statistics heartbeat. Every five seconds, the XVM's stats collection thread wakes up and gathers metrics from all collectors. It assembles these into a heartbeat message and then simultaneously delivers it through three paths if all are enabled. The heartbeat gets traced to a log file in human-readable format, written to a binary heartbeat log in compact format, and emitted over the SMA heartbeat channel. A monitoring tool subscribed to that channel receives it immediately and updates its real-time dashboard. Days later, when someone needs to analyze that period, the Stats Dump Tool reads the binary log and produces detailed reports—all without touching the running system.

## Operational Trade-offs

### Performance vs Visibility

Every operational capability comes with a performance cost, and Talon's design gives you the controls to balance visibility against overhead. The conservative default configuration collects minimal statistics with low overhead—enough to know your microservices are healthy without impacting latency. In development, you'll typically enable detailed trace logging and per-transaction stats because understanding behavior matters more than performance. Production deployments usually balance binary logging for post-mortem analysis with selective SMA emission of critical metrics for real-time dashboards. When troubleshooting production issues, you can temporarily enable detailed statistics for specific microservices or message types, gathering the visibility you need without permanently increasing overhead across the entire system.

### Real-time vs Historical

Real-time monitoring through SMA channels shows you what's happening right now. Your dashboards update continuously, alerts fire immediately when thresholds are breached, and operators have instant visibility into system behavior. This comes at the cost of requiring messaging infrastructure and continuous processing by monitoring applications.

Historical analysis through binary logs takes a different approach. Transaction logs and heartbeat logs capture everything to disk with minimal overhead, then offline analysis tools read those logs later without any impact on running systems. This requires log retention and storage but gives you the ability to perform arbitrarily expensive analysis after the fact.

Many deployments use a hybrid approach: log everything to binary logs for complete historical record, analyze offline when investigating issues, but emit critical metrics over SMA channels for real-time dashboards. This gives you both immediate visibility into key metrics and the ability to deeply analyze historical data when needed.

### Direct vs Messaging-based Admin

The choice between direct TCP administration and Admin over SMA reflects different operational contexts. Direct TCP provides lower latency and simpler setup—admin tools connect directly to XVMs and get immediate responses. This works well during development or when troubleshooting specific services, but it requires network access to each XVM and doesn't scale well when managing hundreds of distributed services.

Admin over SMA scales naturally to large deployments. Adding more XVMs or monitoring tools doesn't require connection management since everything flows through the messaging infrastructure. It supports passive monitoring where observers can watch administrative traffic without active connections. However, it requires that messaging infrastructure to be running and adds the latency of routing through message buses.

## Operational Sections

For detailed configuration and usage:

* [Administration](/talon/operating-applications/administration) - Admin tools and configuration
* [Monitoring](/talon/operating-applications/monitoring) - Statistics configuration and reference
* [Analysis & Troubleshooting](/talon/operating-applications/analysis-and-troubleshooting) - Logging and analysis tools

## Next Steps

1. Understand [Discovery Model](/talon/concepts-and-architecture/discovery-model) for operational infrastructure
2. Review [Administration](/talon/operating-applications/administration) options
3. Configure [Monitoring](/talon/operating-applications/monitoring) appropriate for your environment
4. Set up [Logging](/talon/operating-applications/analysis-and-troubleshooting) for troubleshooting


# Developing Applications

This section describes how to code, develop and configure Talon applications.

From a developer's perspective, a Talon application is comprised of the following artifacts:

* **Message Model**
  * The application has a single ADM based XML message model shared across all the application's microservices. This model contains the definition of all messages shared across all the application's microservices. The model is used to generate classes that are used as messages exchanged between microservices.
* **Microservices**
  * The application is comprised of one or more microservices. The development artifacts for each microservice are as follows:
    * **State Model** (only for State Replication microservices)
      * A State Replication microservice defines its state in an ADM based XML state model. Each state model is private to the microservice and generates classes that serve to store application data and state. EventSourced microservices store data and state in regular Java objects and so do not need a state model
    * **User Code**
      * User code in a microservice is comprised of the following:
        * The HA Policy Declaration
        * Lifecycle Methods
        * Microservice Initializers
        * Message Filters
        * Message Handlers
* **Configuration**
  * The application has a single DDL based XML configuration model shared across all the application's microservices. This model contains application wide configuration, such as configuration of the underlying messaging bus (see [Messaging Model](/talon/concepts-and-architecture/messaging-model)) and configuration of the platform runtime of each of the application's microservices. The configuration model configures the Talon runtime and, therefore, needs to be provided to the microservices only when the microservices are run.

For more information

* See See [Modeling Messages & State](/talon/developing-applications/modeling-messages-and-state) for information on how to [model messages and state](/talon/developing-applications/modeling-messages-and-state/the-modeling-language)
* See [The Code Generator](/talon/developing-applications/modeling-messages-and-state/the-code-generator) on how to operate the ADM code generator to convert models to classes
* See See [Authoring User Code](/talon/developing-applications/authoring-user-code) for information on how to author user code in microservices.
* See [Configuring Messaging](/talon/developing-applications/configuring-messaging) for information on how to configure messaging
* See [Configuring the Microservice Runtime](/talon/developing-applications/configuring-the-runtime) for information on how to configure the microservices' runtime.


# Microservice Template

Talon provides two built-in microservice templates that implement different consensus models for maintaining state consistency across clustered microservice instances.

## Available Templates

### [State Replication Template](/talon/developing-applications/microservice-template/state-replication-template)

Uses Talon's State Replication consensus model where state changes are automatically replicated across all instances by the runtime. State is modeled using ADM (Application Data Model) and is transparent to the Talon runtime.

**Best for:** Applications requiring automatic state synchronization with minimal coding effort.

### [Event Sourcing Template](/talon/developing-applications/microservice-template/event-sourcing-template)

Uses Talon's Event Sourcing consensus model where state is maintained by replaying a deterministic sequence of events. State is opaque to the Talon runtime and maintained in your POJO objects.

**Best for:** Applications requiring full control over state management and complex business logic.

## Choosing a Template

Both templates provide high-availability and fault tolerance but differ in how they achieve consistency:

* **State Replication**: Runtime manages state synchronization automatically
* **Event Sourcing**: Application code replays events to rebuild state

For a detailed comparison, see [Consensus Models](/talon/concepts-and-architecture/consensus-models).

## Next Steps

* Review [Consensus Models](/talon/concepts-and-architecture/consensus-models) to understand the differences
* Choose the template that fits your application requirements
* Follow the template-specific guide for implementation details


# State Replication Template

## Overview

State Replication is the simpler of Talon's two High Availability models. With State Replication, Talon automatically replicates changes to your microservice's state and the outbound messages emitted by your message handlers. In the event of failover to a backup or cold start recovery from a transaction log, your microservice's state is available at the same point where processing left off, and the engine will retransmit any outbound messages that were left in doubt as a result of the failure.

### How State Replication Works

With State Replication:

* **State is modeled using ADM** - Define your state using the Application Data Model XML
* **State changes are automatically replicated** - The Talon runtime tracks and replicates all state modifications
* **Transparent state management** - State is managed by the runtime as a tree of POJOs
* **Atomic transactions** - State changes and outbound messages form atomic units of work
* **Automatic failover** - Backup instances maintain synchronized state for seamless takeover

### Key Features

* **Automatic Replication**: Runtime handles state synchronization across cluster members
* **Transparent State Tree**: State organized as an object tree, automatically tracked by runtime
* **ADM-Based Modeling**: State defined using same XML modeling language as messages
* **Built-in Consistency**: Guaranteed state consistency across all instances
* **Transaction Integration**: State changes and message sends are atomic
* **Disk Persistence**: Transaction log enables cold start recovery

### When to Use State Replication

State Replication is ideal when:

* You want automatic state synchronization with minimal code
* State can be modeled using ADM primitives and collections
* You don't need complex custom state logic or inheritance
* Simplicity and ease of development are priorities
* You need guaranteed state consistency across instances

### Comparison with Event Sourcing

| Aspect               | State Replication                         | Event Sourcing                          |
| -------------------- | ----------------------------------------- | --------------------------------------- |
| **State Management** | Runtime manages (transparent)             | Application manages (opaque POJOs)      |
| **State Modeling**   | ADM XML required                          | No modeling required                    |
| **Replication**      | State deltas replicated                   | Inbound messages replicated             |
| **Latency**          | Slightly higher (state tracking overhead) | Lower (no state tracking)               |
| **Complexity**       | Simpler application code                  | More complex (determinism requirements) |
| **Recovery**         | State reconstructed from log              | State reconstructed by message replay   |

See [Consensus Models](/talon/concepts-and-architecture/consensus-models) for detailed comparison.

***

## Building a State Replication Microservice

Creating a State Replication microservice involves five main steps:

1. Model your microservice state using ADM
2. Annotate your main class for State Replication
3. Provide a state factory
4. Write message handlers that operate on state
5. Configure storage (clustering and persistence)

### Step 1: Model Application State

Define your microservice's state using the ADM modeling language in your `model.xml` file. The state model defines entities that form a tree structure with a single root object.

**Example State Model:**

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-ddl"
       namespace="com.example.orderprocessor.state"
       defaultFactoryId="2">

  <entities>
    <!-- Root entity of the state tree -->
    <entity name="OrderBook" id="1">
      <!-- Simple fields -->
      <field name="totalOrders" type="Long" default="0"/>
      <field name="totalValue" type="BigDecimal" default="0.0"/>

      <!-- Collection of entities -->
      <field name="orders" type="Order" collection="Map"
             key="orderId" id="1"/>

      <!-- Nested entity field -->
      <field name="statistics" type="Statistics" id="2"/>
    </entity>

    <!-- Child entity -->
    <entity name="Order" id="2">
      <field name="orderId" type="String"/>
      <field name="symbol" type="String"/>
      <field name="quantity" type="Integer"/>
      <field name="price" type="BigDecimal"/>
      <field name="status" type="String"/>
    </entity>

    <!-- Embedded entity for nested data -->
    <entity name="Statistics" id="3" embedded="true">
      <field name="ordersProcessed" type="Long" default="0"/>
      <field name="rejectCount" type="Long" default="0"/>
    </entity>
  </entities>
</model>
```

**Important Modeling Considerations**:

* The state tree must have a single root entity
* Entities can contain simple fields, entity fields, and collections
* See [State Tree Limitations](#state-tree-limitations) for restrictions
* See [The Modeling Language](/talon/developing-applications/modeling-messages-and-state/the-modeling-language) for complete modeling guide

### Step 2: Annotate Main Class for State Replication

Use the `@AppHAPolicy` annotation to declare that your microservice uses State Replication:

```java
package com.example.orderprocessor;

import com.neeve.aep.AepEngine;
import com.neeve.aep.annotations.AppHAPolicy;

@AppHAPolicy(value = AepEngine.HAPolicy.StateReplication)
public class OrderProcessorApp {
    // ... application code ...
}
```

### Step 3: Provide a State Factory

The AEP engine needs to be able to create your application's initial state. Provide a state factory method annotated with `@AppStateFactoryAccessor`:

```java
import com.neeve.aep.IAepApplicationStateFactory;
import com.neeve.aep.annotations.AppStateFactoryAccessor;
import com.neeve.sma.MessageView;

@AppStateFactoryAccessor
final public IAepApplicationStateFactory getStateFactory() {
    return new IAepApplicationStateFactory() {
        @Override
        final public OrderBook createState(MessageView view) {
            // Return a new, empty state root
            return OrderBook.create();
        }
    };
}
```

{% hint style="warning" %}
**Important**: The state factory should return an empty, uninitialized object. The platform will:

1. Invoke the factory during initialization with a `null` argument to determine the state root type
2. Subsequently invoke it on receipt of a MessageView when state hasn't been created
3. Manage the state tree after creation - don't store references to the state yourself
   {% endhint %}

### Step 4: Write Message Handlers

Message handlers receive both the inbound message and the state root object. The handler updates state and sends outbound messages as an atomic transaction:

```java
import com.neeve.aep.AepMessageSender;
import com.neeve.aep.annotations.EventHandler;
import com.neeve.aep.annotations.AppInjectionPoint;

public class OrderProcessorApp {
    private AepMessageSender messageSender;

    // Inject message sender for outbound messages
    @AppInjectionPoint
    final public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    @EventHandler
    final public void onNewOrder(NewOrderRequest request, OrderBook orderBook) {
        // Create and populate order entity
        Order order = Order.create();
        order.setOrderId(request.getOrderId());
        order.setSymbol(request.getSymbol());
        order.setQuantity(request.getQuantity());
        order.setPrice(request.getPrice());
        order.setStatus("ACCEPTED");

        // Update state - changes are automatically tracked and replicated
        orderBook.getOrders().put(order.getOrderId(), order);
        orderBook.setTotalOrders(orderBook.getTotalOrders() + 1);
        orderBook.setTotalValue(
            orderBook.getTotalValue().add(
                order.getPrice().multiply(
                    BigDecimal.valueOf(order.getQuantity())
                )
            )
        );

        // Update embedded statistics
        orderBook.getStatistics().setOrdersProcessed(
            orderBook.getStatistics().getOrdersProcessed() + 1
        );

        // Send outbound message - atomically replicated with state changes
        OrderConfirmation confirmation = OrderConfirmation.create();
        confirmation.setOrderId(order.getOrderId());
        confirmation.setStatus("ACCEPTED");
        messageSender.sendMessage("confirmations", confirmation);
    }

    @EventHandler
    final public void onCancelOrder(CancelOrderRequest request, OrderBook orderBook) {
        Order order = orderBook.getOrders().get(request.getOrderId());
        if (order != null && "ACCEPTED".equals(order.getStatus())) {
            // Update state
            order.setStatus("CANCELLED");
            orderBook.getStatistics().setRejectCount(
                orderBook.getStatistics().getRejectCount() + 1
            );

            // Send confirmation
            OrderConfirmation confirmation = OrderConfirmation.create();
            confirmation.setOrderId(order.getOrderId());
            confirmation.setStatus("CANCELLED");
            messageSender.sendMessage("confirmations", confirmation);
        }
    }
}
```

**Key Points**:

* State changes and message sends form an atomic transaction
* All modifications to state are automatically tracked and replicated
* No need for manual state serialization or replication code
* State root is passed to every handler method

### Step 5: Configure Storage

Configure storage in your DDL to enable clustering and persistence.

#### Register Object Factories

Both message factories and state factories must be registered with the runtime:

**DDL Configuration:**

```xml
<app name="order-processor" mainClass="com.example.orderprocessor.OrderProcessorApp">
  <messaging>
    <factories>
      <factory name="com.example.orderprocessor.messages.MessageFactory"/>
    </factories>
    <buses>
      <bus name="orders-bus">
        <channels>
          <channel name="orders" join="true"/>
          <channel name="confirmations" join="false"/>
        </channels>
      </bus>
    </buses>
  </messaging>

  <storage enabled="true">
    <factories>
      <factory name="com.example.orderprocessor.state.StateFactory"/>
    </factories>
    <clustering enabled="true">
      <storeName>order-processor-store</storeName>
      <localPort>10000</localPort>
      <discoveryDescriptor>nvds://localhost:4090</discoveryDescriptor>
    </clustering>
    <persistence enabled="true">
      <flushOnCommit>false</flushOnCommit>
      <storeRoot>rdat</storeRoot>
    </persistence>
  </storage>
</app>
```

**Programmatic Registration Alternative:**

```java
@AppInjectionPoint
public void initialize(AepEngine engine) {
    // Register message factory
    engine.registerFactory(new com.example.orderprocessor.messages.MessageFactory());

    // Register state factory for replication
    engine.registerFactory(new com.example.orderprocessor.state.StateFactory());
}
```

#### Enable Clustering

Clustering allows multiple instances to discover each other and form an HA cluster:

```xml
<clustering enabled="true">
  <storeName>order-processor-store</storeName>
  <localPort>10000</localPort>
  <localIfAddr>192.168.1.100</localIfAddr>
  <linkParams>SO_REUSEADDR=true,TCP_NODELAY=true</linkParams>
  <discoveryDescriptor>nvds://localhost:4090</discoveryDescriptor>
  <initWaitTime>5000</initWaitTime>
  <failOnMultiplePrimaries>true</failOnMultiplePrimaries>
  <memberElectionPriority>100</memberElectionPriority>
</clustering>
```

**Key clustering concepts**:

* Instances with same `storeName` form a cluster
* One instance elected as primary via leadership election
* Primary establishes messaging and invokes handlers
* Backups maintain synchronized state for failover

#### Enable Persistence

Persistence logs the replication stream to disk for cold start recovery:

```xml
<persistence enabled="true">
  <flushOnCommit>false</flushOnCommit>
  <autoFlushSize>8192</autoFlushSize>
  <storeRoot>rdat</storeRoot>
  <compaction>
    <compactOnStart>false</compactOnStart>
    <compactionThreshold>1024</compactionThreshold>
  </compaction>
</persistence>
```

{% hint style="info" %}
**Clustering Requires Persistence**: When clustering is enabled, persistence must also be enabled. The transaction log is used to initialize new cluster members that connect to the primary.
{% endhint %}

See [Storage Configuration](/talon/reference/configuration#storage-configuration) for complete configuration reference.

***

## State Tree Limitations

State Replication has several important restrictions that developers must understand. These are documented in detail in [Programming Fundamentals - State Tree Limitations](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals#state-tree-limitations):

### Key Restrictions

1. **Single Parent Restriction**: An entity can only appear in one location in the state tree (throws `IllegalStateException` if violated)
2. **No Multiple Fields of Same Type**: Cannot have multiple non-embedded entity fields of the same type in a parent
3. **No State Tree Cycles**: Cycles (including self-references) are not supported
4. **No Primitive Collections**: Collections must use boxed types (e.g., `Integer` not `int`)
5. **No Inheritance**: Inheritance not supported; use entity inlining for polymorphism

### Workarounds

**For Single Parent Restriction**: Store entities in a Map and reference by ID:

```java
// Instead of storing Customer in multiple places:
// order1.setCustomer(customer);  // Not allowed
// order2.setCustomer(customer);  // Would throw exception

// Use a Map and reference by ID:
orderBook.getCustomers().put(customer.getId(), customer);
order1.setCustomerId(customer.getId());
order2.setCustomerId(customer.getId());
```

See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals#state-tree-limitations) for complete details and additional workarounds.

***

## Related Documentation

### Core Concepts

* [Consensus Models](/talon/concepts-and-architecture/consensus-models) - Understanding State Replication vs Event Sourcing
* [Transactions](/talon/concepts-and-architecture/transactions) - How transactions work with state replication
* [Application Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - State creation and initialization

### Development

* [Modeling Messages & State](/talon/developing-applications/modeling-messages-and-state) - Complete ADM modeling guide
* [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) - State tree limitations and restrictions
* [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Writing message handlers

### Configuration

* [Storage Configuration](/talon/reference/configuration#storage-configuration) - Complete storage configuration reference
* [Configuring Threading](/talon/developing-applications/configuring-the-runtime/threading) - Optimize replication performance

### Operations

* [Transaction Log Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/transaction-log-tool) - Browse and analyze transaction logs
* [Querying Transaction Logs](/talon/operating-applications/analysis-and-troubleshooting/querying-transaction-logs) - XPQL query language

***

## Next Steps

1. **Start modeling**: Define your state in `model.xml` following ADM syntax
2. **Create application class**: Annotate with `@AppHAPolicy` and provide state factory
3. **Write handlers**: Implement message handlers that operate on state
4. **Configure storage**: Enable clustering and persistence in DDL
5. **Test failover**: Verify state consistency and message retransmission
6. **Tune performance**: Adjust threading and batching based on load testing
7. **Monitor in production**: Track transaction log size and replication latency


# Event Sourcing Template

## Overview

Event Sourcing is Talon's High Availability model that provides the best performance for latency-sensitive applications. With Event Sourcing, Talon replicates inbound messages rather than state changes. Each instance processes the same sequence of messages deterministically to maintain identical state and produce identical outputs.

### How Event Sourcing Works

With Event Sourcing:

* **Inbound messages are replicated** - Messages are replicated to backup instances in parallel with processing
* **State is application-managed** - Your state is stored in POJOs, opaque to the Talon runtime
* **Deterministic processing ensures consistency** - All instances execute identical code on identical inputs
* **No state modeling required** - State can be any Java objects (ADM modeling optional)
* **Lower replication overhead** - Messages replicated in serialized form without re-serialization

### Key Features

* **Application-Controlled State**: Complete control over state structure and management
* **Opaque State**: Runtime doesn't inspect or manage your state
* **Low Latency**: Messages replicated in parallel with handler execution
* **Efficient Replication**: Received messages already serialized, no re-serialization cost
* **Full Flexibility**: Use any Java objects for state, no ADM constraints
* **Deterministic Recovery**: State reconstructed through message replay

### When to Use Event Sourcing

Event Sourcing is ideal when:

* Ultra-low latency is critical (sub-millisecond requirements)
* You need full control over state management
* State logic is complex or involves custom algorithms
* Your state doesn't fit ADM modeling constraints
* You want to use standard Java POJOs
* Replication overhead must be minimized

### Comparison with State Replication

| Aspect                  | Event Sourcing                          | State Replication                  |
| ----------------------- | --------------------------------------- | ---------------------------------- |
| **State Management**    | Application manages (opaque POJOs)      | Runtime manages (transparent)      |
| **State Modeling**      | No modeling required                    | ADM XML required                   |
| **Replication**         | Inbound messages replicated             | State deltas replicated            |
| **Latency**             | Lower (parallel replication)            | Slightly higher (state tracking)   |
| **Complexity**          | More complex (determinism requirements) | Simpler application code           |
| **Recovery**            | State reconstructed by message replay   | State reconstructed from log       |
| **Discipline Required** | High (must avoid divergence)            | Low (runtime enforces consistency) |

See [Consensus Models](/talon/concepts-and-architecture/consensus-models) for detailed comparison.

***

## Building an Event Sourcing Microservice

Creating an Event Sourcing microservice involves five main steps:

1. Model your application messages using ADM
2. Annotate your main class for Event Sourcing
3. Manage microservice state in POJOs
4. Write deterministic message handlers
5. Configure storage (clustering and persistence)

### Step 1: Model Application Messages

Define your microservice's messages using the ADM modeling language in your `model.xml` file. Unlike State Replication, you don't need to model your state in ADM - only your messages.

**Example Message Model:**

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-ddl"
       namespace="com.example.orderprocessor.messages"
       defaultFactoryId="1">

  <messages>
    <!-- Inbound message -->
    <message name="NewOrderRequest" id="1">
      <field name="orderId" type="String"/>
      <field name="symbol" type="String"/>
      <field name="quantity" type="Integer"/>
      <field name="price" type="BigDecimal"/>
    </message>

    <!-- Outbound message -->
    <message name="OrderConfirmation" id="2">
      <field name="orderId" type="String"/>
      <field name="status" type="String"/>
      <field name="timestamp" type="Long"/>
    </message>

    <message name="CancelOrderRequest" id="3">
      <field name="orderId" type="String"/>
    </message>
  </messages>
</model>
```

See [The Modeling Language](/talon/developing-applications/modeling-messages-and-state/the-modeling-language) for complete modeling guide.

### Step 2: Annotate Main Class for Event Sourcing

Use the `@AppHAPolicy` annotation to declare that your microservice uses Event Sourcing:

```java
package com.example.orderprocessor;

import com.neeve.aep.AepEngine;
import com.neeve.aep.annotations.AppHAPolicy;

@AppHAPolicy(value = AepEngine.HAPolicy.EventSourcing)
public class OrderProcessorApp {
    // ... application code ...
}
```

### Step 3: Manage Application State

With Event Sourcing, your state is completely private to your application. You can use any Java objects - POJOs, collections, or even third-party data structures:

```java
public class OrderProcessorApp {
    // Private POJO state - opaque to Talon
    private class OrderBook {
        private long totalOrders = 0;
        private BigDecimal totalValue = BigDecimal.ZERO;
        private final Map<String, Order> orders = new HashMap<>();

        public void addOrder(Order order) {
            orders.put(order.getOrderId(), order);
            totalOrders++;
            totalValue = totalValue.add(
                order.getPrice().multiply(BigDecimal.valueOf(order.getQuantity()))
            );
        }

        public Order getOrder(String orderId) {
            return orders.get(orderId);
        }

        public long getTotalOrders() {
            return totalOrders;
        }
    }

    // Simple POJO for order state
    private static class Order {
        private String orderId;
        private String symbol;
        private int quantity;
        private BigDecimal price;
        private String status;

        // Constructor, getters, setters...
    }

    // Application state instance
    private final OrderBook orderBook = new OrderBook();
}
```

**Important Points**:

* State is completely private to your application
* No ADM modeling required for state
* Runtime never inspects or manages your state
* State must be reconstructed deterministically on recovery through message replay

### Step 4: Write Deterministic Message Handlers

Message handlers must be deterministic - they must produce identical state and outputs given identical inputs. Handlers receive only the inbound message (not state, since state is private).

```java
import com.neeve.aep.AepMessageSender;
import com.neeve.aep.AepEngine;
import com.neeve.aep.annotations.EventHandler;
import com.neeve.aep.annotations.AppInjectionPoint;

public class OrderProcessorApp {
    private AepMessageSender messageSender;
    private AepEngine engine;
    private final OrderBook orderBook = new OrderBook();

    // Inject message sender for outbound messages
    @AppInjectionPoint
    final public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    // Inject engine for accessing engine time
    @AppInjectionPoint
    final public void initialize(AepEngine engine) {
        this.engine = engine;
        // Register message factory
        engine.registerFactory(new com.example.orderprocessor.messages.MessageFactory());
    }

    @EventHandler
    final public void onNewOrder(NewOrderRequest request) {
        // Create order from request - deterministic
        Order order = new Order();
        order.setOrderId(request.getOrderId());
        order.setSymbol(request.getSymbol());
        order.setQuantity(request.getQuantity());
        order.setPrice(request.getPrice());
        order.setStatus("ACCEPTED");

        // Update private POJO state
        orderBook.addOrder(order);

        // Send outbound message
        OrderConfirmation confirmation = OrderConfirmation.create();
        confirmation.setOrderId(order.getOrderId());
        confirmation.setStatus("ACCEPTED");
        // IMPORTANT: Use engine.getEngineTime() for deterministic time
        confirmation.setTimestamp(engine.getEngineTime());
        messageSender.sendMessage("confirmations", confirmation);
    }

    @EventHandler
    final public void onCancelOrder(CancelOrderRequest request) {
        Order order = orderBook.getOrder(request.getOrderId());
        if (order != null && "ACCEPTED".equals(order.getStatus())) {
            // Update state deterministically
            order.setStatus("CANCELLED");

            // Send confirmation
            OrderConfirmation confirmation = OrderConfirmation.create();
            confirmation.setOrderId(order.getOrderId());
            confirmation.setStatus("CANCELLED");
            confirmation.setTimestamp(engine.getEngineTime());
            messageSender.sendMessage("confirmations", confirmation);
        }
    }
}
```

**Critical: Preventing Divergence**

Event Sourcing requires special discipline to avoid state divergence between primary and backup:

{% hint style="danger" %}
**Never use non-deterministic operations in handlers:**

* `System.currentTimeMillis()` - Use `engine.getEngineTime()` instead
* `System.nanoTime()` - Not replicated, will cause divergence
* Random numbers - Will differ on backup
* External I/O or database calls - Results not replicated
* Environment variables - May differ between instances
* Clock-dependent logic - Clocks may be skewed
  {% endhint %}

**Safe Practices**:

* **Use `engine.getEngineTime()`** - Provides deterministic timestamp from message
* **Process only message data** - All decisions based on message content and existing state
* **Use message injection** - Inject results from external operations as new messages
* **Use environment replication** - Tunnel environment data into replication stream (see below)

### Step 5: Configure Storage

Configure storage in your DDL to enable clustering and persistence.

#### Register Message Factories

Only message factories need to be registered (not state factories, since state is opaque):

**DDL Configuration:**

```xml
<app name="order-processor" mainClass="com.example.orderprocessor.OrderProcessorApp">
  <messaging>
    <factories>
      <factory name="com.example.orderprocessor.messages.MessageFactory"/>
    </factories>
    <buses>
      <bus name="orders-bus">
        <channels>
          <channel name="orders" join="true"/>
          <channel name="confirmations" join="false"/>
        </channels>
      </bus>
    </buses>
  </messaging>

  <storage enabled="true">
    <clustering enabled="true">
      <storeName>order-processor-store</storeName>
      <localPort>10000</localPort>
      <discoveryDescriptor>nvds://localhost:4090</discoveryDescriptor>
    </clustering>
    <persistence enabled="true">
      <flushOnCommit>false</flushOnCommit>
      <storeRoot>rdat</storeRoot>
    </persistence>
  </storage>
</app>
```

**Programmatic Registration Alternative:**

```java
@AppInjectionPoint
public void initialize(AepEngine engine) {
    // Register message factory for deserialization
    engine.registerFactory(new com.example.orderprocessor.messages.MessageFactory());
}
```

#### Enable Clustering

Clustering allows multiple instances to discover each other and form an HA cluster:

```xml
<clustering enabled="true">
  <storeName>order-processor-store</storeName>
  <localPort>10000</localPort>
  <localIfAddr>192.168.1.100</localIfAddr>
  <linkParams>SO_REUSEADDR=true,TCP_NODELAY=true</linkParams>
  <discoveryDescriptor>nvds://localhost:4090</discoveryDescriptor>
  <initWaitTime>5000</initWaitTime>
  <failOnMultiplePrimaries>true</failOnMultiplePrimaries>
  <memberElectionPriority>100</memberElectionPriority>
</clustering>
```

**Key clustering concepts**:

* Instances with same `storeName` form a cluster
* One instance elected as primary via leadership election
* Primary establishes messaging and invokes handlers
* Backups receive replicated messages and process them identically

#### Enable Persistence

Persistence logs the message stream to disk for cold start recovery:

```xml
<persistence enabled="true">
  <flushOnCommit>false</flushOnCommit>
  <autoFlushSize>8192</autoFlushSize>
  <storeRoot>rdat</storeRoot>
  <compaction>
    <compactOnStart>false</compactOnStart>
    <compactionThreshold>1024</compactionThreshold>
  </compaction>
</persistence>
```

{% hint style="info" %}
**Clustering Requires Persistence**: When clustering is enabled, persistence must also be enabled. The transaction log is used to initialize new cluster members that connect to the primary.
{% endhint %}

{% hint style="warning" %}
**Cold Start Recovery**: With Event Sourcing, recovery from a cold start (no backup running) requires replaying the entire inbound message stream from disk. For long-running applications with high message volumes, transaction log compaction and checkpointing strategies are critical.
{% endhint %}

See [Storage Configuration](/talon/reference/configuration#storage-configuration) for complete configuration reference.

***

## Advanced Techniques

### Using Engine Time

The `AepEngine.getEngineTime()` method provides deterministic timestamps for Event Sourcing applications:

```java
private volatile AepEngine engine;

@AppInjectionPoint
public void initialize(AepEngine engine) {
    this.engine = engine;
}

@EventHandler
public void onMessage(TradeUpdate message) {
    OrderEvent event = OrderEvent.create();

    // Engine time is identical on primary and backup
    event.setTimeReceivedAsTimestamp(engine.getEngineTime());

    messageSender.sendMessage("order-events", event);
}
```

**How It Works**:

* When called from a handler in Event Sourcing mode, returns timestamp from message event
* Timestamp is captured just before handler dispatch
* Same timestamp replicated to backup, ensuring deterministic processing
* Outside handlers or in State Replication mode, returns `System.currentTimeMillis()`

### Message Injection

Message injection allows you to inject messages into the processing stream for fault-tolerant execution. This is useful for:

* **Deferred processing** - Schedule work to be done later
* **Async operations** - Inject results from other threads
* **External system integration** - Inject data from external sources

**Example: Scheduling Deferred Work**

```java
@EventHandler
public void onNewOrder(NewOrderRequest request) {
    // Process order immediately
    processOrder(request);

    // Schedule timeout check for 30 seconds later
    OrderTimeoutCheck timeoutCheck = OrderTimeoutCheck.create();
    timeoutCheck.setOrderId(request.getOrderId());

    // Inject with 30-second delay - replicated to backup
    engine.injectMessage(timeoutCheck, 30000);
}

@EventHandler
public void onOrderTimeoutCheck(OrderTimeoutCheck check) {
    // This executes 30 seconds later on both primary and backup
    Order order = orderBook.getOrder(check.getOrderId());
    if (order != null && "PENDING".equals(order.getStatus())) {
        // Cancel timed-out order
        cancelOrder(order);
    }
}
```

**Example: Integrating External Operations**

```java
@EventHandler
public void onHostnameRequest(BroadcastHostNameRequest request) {
    // Get hostname from local environment (non-deterministic)
    String hostname = InetAddress.getLocalHost().getHostName();

    // Inject result as a message (now deterministic)
    SendHostNameCommand command = SendHostNameCommand.create();
    command.setHostName(hostname);
    engine.injectMessage(command);
}

@EventHandler
public void onSendHostName(SendHostNameCommand command) {
    // This handler processes the injected message
    // Both primary and backup receive the same hostname
    BroadcastHostNameResponse response = BroadcastHostNameResponse.create();
    response.setHostName(command.getHostName());
    messageSender.sendMessage("hostname-response", response);
}
```

See [Scheduling Messages](/talon/developing-applications/authoring-user-code/message-injection) for complete message injection documentation.

### Environment Replication

Environment Replication allows you to tunnel local environment data into the replication stream without creating separate transactions. This is more efficient than message injection for frequently-accessed environment data.

**Example: Hostname Provider**

```java
private HostNameEnvironmentProvider hostNameProvider = new HostNameEnvironmentProvider();
private XString hostName = XString.create(256, true, true);

@AppInjectionPoint
public void initialize(AepEngine engine) {
    // Register environment provider
    hostNameProvider.register(engine);
}

@EventHandler
public void onMessage(BroadcastHostNameRequest message) {
    BroadcastHostNameResponse response = BroadcastHostNameResponse.create();

    // Call environment provider - captures or replays hostname
    hostNameProvider.getHostNameTo(hostName);

    // Set hostname in response and send
    response.setHostName(hostName);
    messageSender.sendMessage("hostname-response", response);
}
```

**How It Works**:

* On primary: Provider records hostname lookup in replication buffer
* On backup: Provider replays recorded hostname from buffer
* All within same transaction - no additional message injection overhead

***

## Deterministic Programming Rules

Event Sourcing requires strict adherence to deterministic programming rules:

### What You MUST Do

1. **Base all decisions on message data and existing state only**
2. **Use `engine.getEngineTime()` for all time-dependent logic**
3. **Keep handlers synchronous and single-threaded**
4. **Use message injection for async operations**
5. **Use environment replication for environment data**
6. **Ensure identical handler execution on all instances**

### What You MUST NOT Do

1. **Never call `System.currentTimeMillis()` or `System.nanoTime()`**
2. **Never use `Random` or any non-deterministic algorithms**
3. **Never read from files, databases, or external systems directly**
4. **Never use thread-local storage or instance variables modified outside handlers**
5. **Never depend on environment variables or system properties in handlers**
6. **Never use object identity (e.g., `System.identityHashCode()`) for logic**

See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) for complete deterministic programming rules.

***

## Related Documentation

### Core Concepts

* [Consensus Models](/talon/concepts-and-architecture/consensus-models) - Understanding Event Sourcing vs State Replication
* [Transactions](/talon/concepts-and-architecture/transactions) - How transactions work with Event Sourcing
* [Application Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Initialization and recovery

### Development

* [Modeling Messages & State](/talon/developing-applications/modeling-messages-and-state) - ADM modeling for messages
* [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) - Deterministic coding rules
* [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Writing message handlers
* [Scheduling Messages](https://github.com/neeveresearch/nvx-docs/blob/master/talon/developing-applications/authoring-user-code/message-processing/scheduling-messages.md) - Message injection

### Configuration

* [Storage Configuration](/talon/reference/configuration#storage-configuration) - Complete storage configuration reference
* [Configuring Threading](/talon/developing-applications/configuring-the-runtime/threading) - Optimize replication performance

### Operations

* [Transaction Log Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/transaction-log-tool) - Browse and analyze transaction logs
* [Querying Transaction Logs](/talon/operating-applications/analysis-and-troubleshooting/querying-transaction-logs) - XPQL query language

***

## Next Steps

1. **Start modeling**: Define your messages in `model.xml` following ADM syntax
2. **Create application class**: Annotate with `@AppHAPolicy(EventSourcing)` and design state POJOs
3. **Write handlers**: Implement deterministic message handlers
4. **Configure storage**: Enable clustering and persistence in DDL
5. **Test failover**: Verify state consistency through message replay
6. **Test determinism**: Ensure identical processing on multiple instances
7. **Monitor in production**: Track transaction log size and replay time


# Modeling Messages & State

This section describes how to model the messages and state for your Talon microservice using the ADM (Application Data Model) language and generate Java classes from those models.

## Overview

Talon microservices work with strongly-typed POJOs (Plain Old Java Objects) for both messages and state. Rather than hand-writing these classes, you define them using an XML-based modeling language, and Talon's code generator converts your models into optimized Java classes.

This approach provides several benefits:

* **Type safety**: Compile-time checking of message and state access
* **Performance**: Generated code is optimized for zero-garbage operation
* **Simplicity**: Focus on your data model, not Java boilerplate
* **Serialization**: Automatic wire format handling for messaging
* **Evolution**: Built-in support for schema versioning and evolution

## What You'll Learn

This section covers two main topics:

### [The ADM Modeling Language](/talon/developing-applications/modeling-messages-and-state/the-modeling-language)

Learn how to define your application's data model using XML:

* **Message modeling**: Define inbound and outbound message types
* **State modeling**: Model your microservice's persistent state as entity trees
* **Type system**: Use built-in types (primitives, strings, decimals) and custom types
* **Collections**: Model arrays, lists, sets, and maps
* **Field properties**: Configure nullability, defaults, and constraints
* **Annotations**: Add metadata to control code generation
* **Model composition**: Import and reuse models across projects

### [Running the Code Generator](/talon/developing-applications/modeling-messages-and-state/the-code-generator)

Integrate Talon's code generator into your build process:

* **Build integration**: Use with Maven, Gradle, or Ant
* **Generator configuration**: Control code generation options
* **Encoding types**: Choose wire format optimizations
* **Generated artifacts**: Understand the generated Java classes
* **Compilation**: Compile generated code with your application

## Quick Start

A typical workflow for modeling and code generation:

1. **Define your model** in an XML file (e.g., `src/main/model/model.xml`):

   ```xml
   <model xmlns="http://www.neeveresearch.com/schema/x-ddl">
     <messages>
       <message name="OrderRequest">
         <field name="orderId" type="String"/>
         <field name="quantity" type="Integer"/>
       </message>
     </messages>

     <entities>
       <entity name="OrderBook">
         <field name="orders" type="OrderRequest" collection="List"/>
       </entity>
     </entities>
   </model>
   ```
2. **Configure the code generator** in your build file (Maven example):

   ```xml
   <plugin>
     <groupId>com.neeve</groupId>
     <artifactId>nvx-platform-maven-plugin</artifactId>
     <executions>
       <execution>
         <goals>
           <goal>adm</goal>
         </goals>
       </execution>
     </executions>
   </plugin>
   ```
3. **Build your project** - Generated POJOs are created automatically:

   ```bash
   mvn compile
   ```
4. **Use the generated classes** in your message handlers:

   ```java
   @EventHandler
   public void onOrderRequest(OrderRequest order, OrderBook orderBook) {
     orderBook.getOrders().add(order);
   }
   ```

## Related Topics

* [Message Processing](/talon/developing-applications/authoring-user-code/message-processing) - Using generated message classes in handlers
* [Introduction](/talon/introduction) - See the simple example of modeling and code generation

## Next Steps

1. Read [The ADM Modeling Language](/talon/developing-applications/modeling-messages-and-state/the-modeling-language) to understand the full modeling capabilities
2. Learn about [Running the Code Generator](/talon/developing-applications/modeling-messages-and-state/the-code-generator) to integrate into your build
3. Explore [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type) for wire format optimization


# The Modeling Language

Talon aims to make working with messages and state as simple as working with Plain Old Java Objects (POJOs). The **Application Data Modeler** (ADM) defines an XML language to model messages and state entities and tools to generate message and state classes. The generated code handles encoding, serialization and transactional functionality that underpin the platform. This section describes how to model messages and state and to use the code generation tools.

## Overview

The following depicts the various sections of an ADM model.

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-adml"
       xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
       namespace="com.example.trading"
       name="MyModel"
       doc="My model"
       defaultFactoryId="100" >
     
  <import model="../other/model.xml"/>
  <import model="com/mynamespace/messages.xml"/>
  <factories>
    <factory name="MyObjectFactory" id="1" />
  </factories>
 
  <enumerations>
    <!-- Enumerations -->
  </enumerations>
 
  <types>
    <!-- Semantic types -->
  </types>
 
  <fields>
    <!-- reusable field definitions -->
  </fields>
 
  <messages>
    <!-- Messages -->
  </messages>
 
  <entities>
    <!-- Entity Definitions -->
  </entities>
 
  <collections>
    <!-- Collection Definitions -->
  </collections>
</model>
```

### Messages

The `messages` section contains message definitions.

* Each message is assigned a numeric id.
  * Message ids must be unique across all messages, entities and collections in a model and should only be reused over the lifetime of the model per the ADM model versioning rules.
* Each message is comprised of a set of fields.
* Each field is of a type supported by the [ADM Type System](#the-adm-type-system).
* Each field is assigned a numeric id.
  * Field ids must be unique within a message definition and should only be reused over the lifetime of a message definition per the ADM model versioning rules.

The following defines an `AddCustomerMessage` message. It contains the following fields

* A field named `firstName` of type `String` with an id of 1.
* A field named `lastName` of type `String` with an id of 2
* A field name `age` of type `Integer` with an id of 3
* A field named `address` of type `Address` with an id of 4.
  * `Address` is an Embedded Entity type

{% hint style="info" %}
The `state` and `city` fields in the `Address` entity are enumerations whose definitions are not depicted in the example below.
{% endhint %}

```xml
    <messages>
        <message name="AddCustomerMessage" id="1">
            <field name="firstName" type="String" id="1"/>
            <field name="lastName" type="String" id="2"/>
            <field name="age" type="String" id="3"/>
            <field name="address" type="Address" id="4"/>
            .
            .
            .
        </message>
    </messages>
   
    <entities>
        <entity name="Address" id="100" asEmbedded="true">
            <field name="streetNumber" type="Integer" id="1"/>
            <field name="streetName" type="String" id="2"/>
            <field name="city" type="City" id="3"/>
            <field name="state" type="State" id="4"/>
            <field name="zipCode" type="Integer" id="5"/>
            .
            .
            .
        </entity>
   </entities>
```

{% hint style="info" %}
See [Fields](#fields-1) for more information on the various attributes in the `field` element
{% endhint %}

Each message definition generates a Java class. The class's namespace is the model's namespace, and the class name is the name of the message in the model. The following is a code snippet depicting how the generated message is instantiated and populated.

```java
// Code that creates and populates the message
AddCustomerMessage message = AddCustomerMessage.create();
message.setFirstName("John");
message.setLastName("Doe");
message.setAge(42);
Address address = Address.create();
address.setStreetNumber(42);
address.setStreetName("Doe Lane");
address.setCity(City.SomeCity);
address.setState(State.XY);
address.setZipCode(12456);
message.setAddress(address);
```

See [Messages](#messages-1) for more information

### Entities

The `entities` section contains entity definitions. There are two types of entities:

* State Entities
* Embedded Entities

Each entity definition supports an `asEmbedded` attribute. An entity with `asEmbedded=false` is a state entity while an entity with `asEmbedded=true` is an embedded entity.

#### Embedded Entities

An embedded entity is a field container just like a message in that it can contain fields of types from the [ADM Type System](#the-adm-type-system). Embedded entities are technically part of the [ADM Type System](#the-adm-type-system) and, as such, can contain fields that reference other embedded entities as the field type. Embedded entities cannot contain fields that reference other state entities and collections as the field type.

#### State Entities

State entities are used to model a microservice's state tree. A state entity is a field container just like a message or embedded entity except that state entities can contain fields that reference other State entities and [collections](#collections) as the field type. These references to other state entities and [collections](#collections) form the parent child relationships in the modeled state tree.

The following is an example of a state tree modeled using ADM

```xml
    <entities>
        <entity name="Store" id="1">
            <field name="customers" type="Customers" id="1"/>
            .
            .
            .
        </entity>
        
        <entity name="Customer" id="2">
            <field name="firstName" type="String" id="1"/>
            <field name="lastName" type="String" id="2"/>
            <field name="age" type="String" id="3"/>
            <field name="address" type="Address" id="4"/>
            .
            .
            .
        </entity>
        
        <entity name="Order" id="3">
            <field name="orderId" type="String" id="1"/>
            <field name="terms" type="PaymentTerms" id="2"/>
            .
            .
            .
        </entity>
                
        <entity name="Address" id="100" asEmbedded="true">
            <field name="streetNumber" type="Integer" id="1"/>
            <field name="streetName" type="String" id="2"/>
            <field name="city" type="City" id="3"/>
            <field name="state" type="State" id="4"/>
            <field name="zipCode" type="Integer" id="5"/>
            .
            .
            .
        </entity>
    </entities>
```

In the above, the `Customer` entity contains an embedded `Address` entity and references a child `Order` entity. The `Customer`-`Order` relationship is a parent-child relationship in the state model.

Setting and getting referenced state entities is the same as with embedded entities. Here is an example that illustrates the creation and population of a `Customer` entity object

```
Customer customer = Customer.create();
customer.setFirstName("John");
customer.setLastName("Doe");
customer.setAge(42);
Address address = Address.create();
address.setStreetNumber(42);
address.setStreetName("Doe Lane");
address.setCity(City.SomeCity);
address.setState(State.XY);
address.setZipCode(12456);
customer.setAddress(address);
Order order = Order.create();
order.setOrderId(5);
order.setTerms(PaymentTerms.NET30);
customer.setOrder(order);
store.getCustomers().put(100, customer); <-- the code that adds the customer to the collection
```

#### Difference Between a State and Embedded Entity

The difference between a state entity and embedded entity is related to object serialization. When a message, state entity or embedded entity containing another embedded entity is serialized, then the contents of container message, state entity or embedded entity and the contained embedded entity are all serialized together into the same unit of transportation and/or persistence. However, when a state entity A contains a field that references another state entity B as uts type, then when A is serialized, it only serializes the contents of A and not B. In other words, a state entity containing a field that refers to another state entity is a referential relationship while a field of an embedded entity type is a containment relationship.

In the above example, if the `Customer` object was serialized, the serialized contents would contain the `Customer` fields and the `Address` fields. If the serialized contents were used to materialize a new `Customer` object, then the `Address` field would also be present in the materialized `Customer`. However, when the `Customer` object is serialized, the serialized contents do not contain the serialized form of the `Order` object i.e. the `order` field would be null in the `Customer` object materialized from the serialized contents. In other words, when the state tree is persisted, each node in the tree is persisted independently and the relationships reconstituted when deserialized from the persisted form. The storing and reconstituting of the relationships between the objects is done by the Talon runtime during state persistence and replication so that the application always works with the fully constituted object tree.

See [Entities](#entities-1) for more information

### Collections

The `Collections` section contain collection definitions. A `Collection` is a collection of State Entities of one of the following types

* Queue
* LongMap
* StringMap

`Collections` are part of a state tree model. It serves as a node in the state tree that is a child node of the State Entity that references the Collection and is the parent node of each State Entity that it contains. Here is an example to illustrate this

```xml
    <entities>
        <entity name="Customer" id="1">
            <field name="id" type="Long" id="1" isKey="true"/>
            <field name="firstName" type="String" id="2"/>
            <field name="lastName" type="String" id="3"/>
            <field name="age" type="String" id="4"/>
            <field name="address" type="Address" id="5"/>
            .
            .
            .
        </entity>
        
        <entity name="Order" id="2">
            <field name="orderId" type="String" id="1"/>
            <field name="terms" type="PaymentTerms" id="2"/>
            .
            .
            .
        </entity>
                
        <entity name="Address" id="100" asEmbedded="true">
            <field name="streetNumber" type="Integer" id="1"/>
            <field name="streetName" type="String" id="2"/>
            <field name="city" type="City" id="3"/>
            <field name="state" type="State" id="4"/>
            <field name="zipCode" type="Integer" id="5"/>
            .
            .
            .
        </entity>
    </entities>
    
    <collections>
        <collection name="Customers" is="LongMap" contains="Customer" id="1000"/>
    </collections>
```

The above model is the same as the prior model but extended to hold a collection of `Customer` state entities in a map keyed by the customer id of type Long. The following code illustrates how to work with code generated from the above.

See [Collections](#collections-1) for more information

### Factories

Message and Object factories are used by the Talon runtime to convert message and state POJOs to and from its serialized form that is transported on the wire and/or persisted to disk. Talon provides programmatic and configuration mechanisms by which message and object factories are registered with Talon SMA and ODS runtimes for this purpose.

The `factories` section defines the various factories generated by the model. Each factory is assigned an id that needs to be unique in the entire system. In addition, each message, state entity and embedded entity is assigned a factory id that must be one of the ids of the factories defined in the `factories` section. The model also contains a `defaultFactoryId` attribute that set the default factory id for those messages and entities that don't have a factory id explicitly assigned to them. The `defaultFactoryId` also needs to be one of the ids of the factories defined in the `factories` section of the model.

### Enumerations

Enumerations are part of the [ADM Type System](#the-adm-type-system). The `enumerations` section contains definitions of the various enumerations in the model. The following example illustrates how an enumeration - the `PaymentTerms` enumeration used by the `terms` field in the `Order` entity - is defined.

```xml
    <enumerations>
        <enum name="PaymentTerms">
            <const name="NET30" value ="0"/>
        </enum>
    </enumerations>

    <entities>
        <entity name="Customer" id="1">
            <field name="id" type="Long" id="1" isKey="true"/>
            <field name="firstName" type="String" id="2"/>
            <field name="lastName" type="String" id="3"/>
            <field name="age" type="String" id="4"/>
            <field name="address" type="Address" id="5"/>
            .
            .
            .
        </entity>
        
        <entity name="Order" id="2">
            <field name="orderId" type="String" id="1"/>
            <field name="terms" type="PaymentTerms" id="2"/>
            .
            .
            .
        </entity>
                
        <entity name="Address" id="100" asEmbedded="true">
            <field name="streetNumber" type="Integer" id="1"/>
            <field name="streetName" type="String" id="2"/>
            <field name="city" type="City" id="3"/>
            <field name="state" type="State" id="4"/>
            <field name="zipCode" type="Integer" id="5"/>
            .
            .
            .
        </entity>
    </entities>
    
    <collections>
        <collection name="Customers" is="LongMap" contains="Customer" id="1000"/>
    </collections>
```

The ADM code generator generates a Java enum class for each enumeration defined in a model.

### Namespace

Each model contains a `namespace` attribute. This value of this attribute is sets the name of the package of the generated classes. For example, in the below model, all the generated classes - `AddCustomerMessage`, `Customer`, `Address`, `PaymentTerms`, `Order` and `Customers` - are in the same `com.mycompany` Java package.

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-adml"
       xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
       namespace="com.mycompany"
       name="MyModel"
       doc="My model"
       defaultFactoryId="100" >
     
  <import model="../other/model.xml"/>
  <import model="com/mynamespace/messages.xml"/>
  <factories>
    <factory name="MyObjectFactory" id="1" />
  </factories>
  
  <messages>
    <message name="AddCustomerMessage" id="1">
        <field name="firstName" type="String" id="1"/>
        <field name="lastName" type="String" id="2"/>
        <field name="age" type="String" id="3"/>
        <field name="address" type="Address" id="4"/>
            .
            .
            .
        </message>
    </messages>
   
    <entities>
        <entity name="Customer" id="1">
            <field name="id" type="Long" id="1" isKey="true"/>
            <field name="firstName" type="String" id="2"/>
            <field name="lastName" type="String" id="3"/>
            <field name="age" type="String" id="4"/>
            <field name="address" type="Address" id="5"/>
            .
            .
            .
        </entity>
        
        <entity name="Order" id="2">
            <field name="orderId" type="String" id="1"/>
            <field name="terms" type="PaymentTerms" id="2"/>
            .
            .
            .
        </entity>
                
        <entity name="Address" id="100" asEmbedded="true">
            <field name="streetNumber" type="Integer" id="1"/>
            <field name="streetName" type="String" id="2"/>
            <field name="city" type="City" id="3"/>
            <field name="state" type="State" id="4"/>
            <field name="zipCode" type="Integer" id="5"/>
            .
            .
            .
        </entity>
    </entities>
    
    <collections>
        <collection name="Customers" is="LongMap" contains="Customer" id="1000"/>
    </collections>
</model>
```

### Imports

Since ADM models are essentially class definitions, Talon aims to enable developers to work with these models as development artifacts equivalent to Java classes. The first part of this integrated support with build tools to inject the code generation seamlessly into the build cycle. The other is the ability to import models as one would do with Java classes.

The `imports` section defines models that contain entity definitions that are referenced in other models. Once a model A is imported into another model B, then entities in B can reference entities in A.

See [Imports](#imports-1) for more information.

### Fields

The ADM model allows fields either to be defined in place (directly on the message or entity that is using the field) or by reference to a field declared in the model's `fields` section. It is a matter of preference which approach an application developer uses: if many messages contain the same field, then it may be more convenient to model the fields in a reusable fashion in the element, but in other cases, it may be more convenient to define the fields in place.

The following is an example that illustrates the use of field definitions>

```xml
<fields>
    <field name="ssn" type="String" id="50"/>
</fields>
 
<entities>
    <entity name="Customer" id="1">
            <field name="id" type="Long" id="1" isKey="true"/>
            <fieldRef ref="ssn"/>
            <field name="firstName" type="String" id="2"/>
            <field name="lastName" type="String" id="3"/>
            <field name="age" type="String" id="4"/>
            <field name="address" type="Address" id="5"/>
            .
            .
            .
        </entity>
</entities>
```

See [Fields](#fields-1) for more information.

### Types

The `types` section is used to define types based on the platform's primitive and built-in types. Such semantic types can be used in place of their corresponding primitive type in the model and will inherit their documentation. The type used for the field in generated messages and entities will be the base type specified by the named field.

The following is an example that illustrates the use of semantic types

```xml
<types>
    <type name="SocialSecurityNumber" base="String" length="12"/>
</types>
 
<fields>
    <field name="ssn" type="SocialSecurityNumber" id="50"/>
</fields>
 
<entities>
    <entity name="Customer" id="1">
            <field name="id" type="Long" id="1" isKey="true"/>
            <fieldRef ref="ssn"/>
            <field name="firstName" type="String" id="2"/>
            <field name="lastName" type="String" id="3"/>
            <field name="age" type="String" id="4"/>
            <field name="address" type="Address" id="5"/>
            .
            .
            .
        </entity>
</entities>
```

See [Semantic Types](#semantic-types) for more information.

## The ADM Type System

ADM supports the following types.

* Primitive Types
* Primitive Type Arrays
* Enumerations
* Enumeration Arrays
* Embedded Entities (Field Groups)
* Embedded Entity Arrays

These types can be used as field types for fields in messages and entity definitions in an ADM model.

### Primitive Types

<table><thead><tr><th width="226">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>Boolean</td><td>boolean</td></tr><tr><td>Byte</td><td>byte</td></tr><tr><td>Char</td><td>char</td></tr><tr><td>Short</td><td>short</td></tr><tr><td>Integer</td><td>int</td></tr><tr><td>Long</td><td>long</td></tr><tr><td>Float</td><td>float</td></tr><tr><td>Double</td><td>double</td></tr><tr><td>String</td><td>java.lang.String</td></tr><tr><td></td><td>com.neeve.lang.XString</td></tr><tr><td>Date</td><td>java.util.Date</td></tr></tbody></table>

### Primitive Type Array

<table><thead><tr><th width="227">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>Boolean[]</td><td>boolean[]<br>com.neeve.lang.XBooleanSequence</td></tr><tr><td>Byte[]</td><td>byte[]<br>com.neeve.lang.XByteSequence</td></tr><tr><td>Char[]</td><td>char[]<br>com.neeve.lang.XCharSequence</td></tr><tr><td>Short[]</td><td>short[]<br>com.neeve.lang.XShortSequence</td></tr><tr><td>Integer[]</td><td>int[]<br>com.neeve.lang.XIntSequence</td></tr><tr><td>Long[]</td><td>long[]<br>com.neeve.lang.XLongSequence</td></tr><tr><td>Float[]</td><td>float[]<br>com.neeve.lang.XFloatSequence</td></tr><tr><td>Double[]</td><td>double]<br>com.neeve.lang.XDoubleSequence</td></tr><tr><td>String[]</td><td>java.lang.String[]<br>com.neeve.lang.XStringSequence</td></tr><tr><td>Date[]</td><td>long[]<br>com.neeve.lang.XDateSequence</td></tr></tbody></table>

### Enumerations

<table><thead><tr><th width="229">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>{enumeration name}</td><td>{model namespace}.{enumeration name}</td></tr></tbody></table>

Enumerations are defined in the `enumerations` section of the ADM model. The name of the enumeration can be used as the type of message and entity fields. See [Enumerations](#enumerations-2) for sample and more details.

### Enumeration Array

<table><thead><tr><th width="228">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>{enumeration name}[]</td><td>{model namespace}.{enumeration name}[]<br>com.neeve.lang.X{Enum Type}Sequence</td></tr></tbody></table>

### Embedded Entity

<table><thead><tr><th width="228">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>{entity name}</td><td>{model namespace}.{entity name}</td></tr></tbody></table>

Embedded entities serve as field groups and can be referenced as a type by fields in entities and messages. Embedded entities are defined in the `entities` section of the ADM model.

### Embedded Entity Array

<table><thead><tr><th width="234">ADM Type</th><th>Java Type</th></tr></thead><tbody><tr><td>{entity name}[]</td><td>{model namespace}.{entity name}[]<br>com.neeve.lang.XLinkedList&#x3C;{model namespace}.{entity name}></td></tr></tbody></table>

## The ADM Model

ADM models are XML models specified by the `x-adml.xsd` schema included at the root of the `nvx-rumi-adm-<version>jar`. Be sure to update your editor's schema validator to reference it.

{% hint style="info" %}
If you are working in an IDE such as Eclipse, try importing the ADM XSD schema into your [eclipse XML catalog](https://wiki.eclipse.org/Using_the_XML_Catalog) so that you can get usage tips on the ADM model by pressing control-space.

{% endhint %}

### Root Element

The root element of a message model is the `model` element, which is used to define a namespace qualified set of modeling elements. To conform to `x-adml.xsd schema` and pass validation, a model must define target XML namespace xmlns="<http://www.neeveresearch.com/schema/x-adml>". This is not to be confused with the `namespace` attribute described below.

The following table describes the various attributes of the root element

<table><thead><tr><th width="178">Attribute</th><th width="484">Description</th><th>Required</th></tr></thead><tbody><tr><td>name</td><td>The Model name can now be specified in the model element itself instead of being supplied externally. If the name contains spaces, then in cases where it used as type name it will be converted to camel case with no spaces (for example "Trading model" would become "TradingModel")</td><td>No</td></tr><tr><td>namespace</td><td>The model's namespace. Model elements use the model namespace as the package name when generating other models, and model importing another model refer to imported model elements using the imported model's namespace.</td><td>Yes</td></tr><tr><td>defaultFactoryId</td><td>The default factory id to be used on model elements that require a factoryId but don't specify one. If the model doesn't define a factory element with a matching id, one is created implicitly by camel casing the model name and appending 'Factory', e.g. TradingFactory.</td><td>No</td></tr><tr><td>doc</td><td>A brief one-line description of the model.</td><td>No</td></tr></tbody></table>

#### Model Name

The ADM model has a name property which is used in certain ways during code generation.

For example, the name of the model is used with the Protobuf code generator as follows:

* As the name of .proto file when IDL is generated.
* As the name of outer java class that wraps model types

The model name can be derived from the model filename, or by defining it explicitly in XML. Name explicitly defined in XML takes precedence over the filename.

{% code title="sampleModel.xml" %}

```xml
<model name="model name goes here">
<!-- Model definitions.....-->
</model>
```

{% endcode %}

For the example above, name property will have a value of *ModelNameGoesHere* (it will be converted to Pascal Notation). If we do not define *name* attribute, the model name would be derived from filename: sampleModel.xml -> *SampleModel*.

{% hint style="info" %}
**Naming the model**

The conversion of a user-defined name to an internally used ADM name can only handle white space as a word separator. Since name will be used to declare a Java class, the developer must take care when specifying a model name to only use white space and characters that are allowed in a class name. \\

If we named the model *sample-model*, the resulting ADM name would be *Sample-model,* which cannot be used as a Java class name because it contains a dash character. Generating code with Protobuf encoding would then cause Java compilation errors. This applies both to name defined through XML attribute and filename-derived ones.
{% endhint %}

### Imports

The import statement allows you to import messages entities and fields from a model in a different namespace.

```xml
<!--Import from the file system-->
<import model="../other/model.xml"/>
 
<!--Or import from the classpath-->
<import model="com/other/model/namespace/model.xml"/>
 
<!-- Types from the imported model can then be referenced -->
<messages>
  <message name="AddCustomerMessage" factoryid="1" id="1">
    <!-- Using a type imported from another model -->
    <field name="address" type="com.other.model.namespace.AddressEntity id="1"/>
    <!-- Using a field definition imported from another model-->
    <fieldRef ref="com.other.model.namespace.firstName"/>
  </message>
</messages>
```

{% hint style="success" %}
**Unqualified Imports Names**

It is not strictly necessary to qualify imported types or fields providing the name is unambiguous across all of a model's imports. However, it is best practice to use the qualified name as it insulates the model from changes to imported models that may cause an unqualified reference to become ambiguous in the future.
{% endhint %}

{% hint style="info" %}
**Mixing encodings via import**

Types from a model imported must be generated with the same encoding type as the model importing them. It is not possible to mix and match different encoding types within a message or entity. So if MessageA is generated with Xbuf and embedded EntityB is generated with Protobuf, Message cannot use EntityB as a field.

When generating code and copying model to output, encoding information is written to target model XML as a directive. If the model for which code is generated and any of its imports have an encoding mismatch, the code generator will raise a model validation error. For imported models that do not have encoding info (resolved directly to OS path or packaged to jar with earlier versions of ADM), this validation is not enforced.
{% endhint %}

See also Imported Model Resolution (TODO)

### Factories

The factories section defines object factories that are used to instantiate the generated object. Each factory in an application must have a unique factory id. The ID is serialized along with the object or transported in MessageMetadata and is used to reconstitute its objects during deserialization. A single model can define multiple factories, allowing Messages and Entities to be grouped together as the application sees fit. A factory can contain a maximum of 32767 types, so in practice, it is rarely a *requirement* to use multiple factories within a single model. Each type in the model defines the factory to which it belongs via its factoryid attribute. The user application may define factories with ids greater than or equal to 1, ids <= 0 are reserved for platform internal use.

```xml
<factories>
  <factory name="CommonFactory" id="1"/>
  <factory name="PizzaServiceFactory" id="2"/>
  <factory name="BookStoreServiceFactory" id="3"/>
</factories>
```

<table><thead><tr><th width="150">Attribute</th><th width="489">Description</th><th>Required</th></tr></thead><tbody><tr><td>name</td><td>The factory name to be used for code generation.</td><td>No</td></tr><tr><td>id</td><td><p>The factory's id which is used to register the factory with the runtime and identify the factory to use when decoding the factory's serialized objects.</p><p>The factory id must be between 1 and 65536 inclusive. Values of 0 or less are reserved for use by the platform.</p></td><td>Yes</td></tr><tr><td>doc</td><td>A brief one-line description for the factory. If more detailed documentation is needed, a child &#x3C;documentation> element can be used.</td><td>No</td></tr><tr><td>deprecated</td><td>When a factory is marked as deprecated it will be marked as deprecated by the code generator.</td><td>No</td></tr></tbody></table>

### Enumerations

Enumeration types can be modeled as follows and can be declared as fields of entity or message types. Unlike other model elements, enumerations aren't tied to a particular factory despite being scoped to the model's namespace. Generated java enumerations don't have dependencies on the rest of the platform and can be used standalone.

#### ADM

```xml
<enumerations>
  <enum name="IntEnumeration" type="int">
    <const name="Value1" value="1"/>
    <const name="Value2" value="2"/>
  </enum>
  <enum name="CharEnumeration" type="char">
    <const name="Value1" value="A"/>
    <const name="Value2" value="B"/>
  </enum>
  <enum name="StringEnumeration" type="string">
    <const name="Value1" value="zero"/>
    <const name="Value2" value="one"/>
  </enum>
</enumerations>
```

Generated Enumerations can specify a type which is accessible in the generated code. The type can be int, char or String. A const should not be removed from an enumeration, if the newly generated code is expected to deserialize enums that were persisted with an earlier version, in addition, the order of constants is important and should not be changed.

#### Generated Source

```java
@Generated(value="com.neeve.adm.AdmEnumeration", date="Fri Jan 23 02:03:22 PST 2015")
public enum IntEnumeration {
  Value1 (1),
  Value2 (2);
  
  /**
   * A zero garbage alternative to {@link Enumeration#values()}.
   */
  final public static List<Enumeration> VALUES = Collections.unmodifiableList(Arrays.asList(Enumeration.values()));
 
  //...
}
```

### Fields

The ADM model allows fields either to be defined in place (directly on the message or entity that is using the field) or by reference to a field declared in the model's section. It is a matter of preference which approach an application developer uses: if many messages contain the same field, then it may be more convenient to model the fields in a reusable fashion in the element, but in other cases, it may be more convenient to define the fields in place.

```xml
<!-- Defines fields which may be referenced by other model elements-->
<fields>
    <field name="myStringField" type="String" id="10000" length="16" doc="My String field" />
</fields>
 
<messages>
    <message name="Message" factoryid="1" id="1">
        <fieldRef  ref="myStringField" name="myField"required="true" />
        <field name="myOtherField" type="String" length="16" id=10001" doc="My Other String field" />
    </message>
</messages>
```

#### Field Element Attributes

<table><thead><tr><th width="135">Attribute</th><th width="520">Description</th><th>Required</th></tr></thead><tbody><tr><td>name</td><td>The name of the field</td><td>yes</td></tr><tr><td>type</td><td>The field type. For message and embedded entities, the supported field types are the types in the <a href="#the-adm-type-system">ADM Type System.</a> For state entities, the field type could also be a name of another state entity or collection.</td><td>yes</td></tr><tr><td>id</td><td><p>The field id</p><p><em><strong>Note:</strong> Field ids must be between 0 and 32767 inclusive</em></p></td><td>no</td></tr><tr><td>required</td><td>Whether or not the field is a required field for the type. Fields declared as required will cause a check to be added for the value being set in the generated types isValid() method.</td><td>no</td></tr><tr><td>isKey</td><td>True if this field should serve as the key when inserting the entity into a Map collection. In this case, insertion of the message in the collection with a given key will update this field with the key value.<br><br><em>Note: Only applies to state entity elements</em></td><td>no</td></tr><tr><td>doc</td><td>The doc to use for the field</td><td>no</td></tr><tr><td>deprecated</td><td>When true all methods generated for the field will be marked as deprecated.</td><td>no</td></tr></tbody></table>

#### Field Reference Element Attributes

<table><thead><tr><th width="134">Attribute</th><th width="521">Description</th><th>Required</th></tr></thead><tbody><tr><td>ref</td><td><p>The name of the referenced field. Name may be with or without the namespace of owner model (i.e. full-qualified name). In both cases, the field will be looked up in both the current model and its imports. If the non-fully-qualified name is declared in the current model and in one of the imports, the field from the current model will take precedence. If multiple imports define the same name, an error will be reported.</p><p>The best practice when importing a field from another model is to use the fully qualified name of the imported field (e.g. com.example.importedmodel.someField), as this insulates your model from future conflicts in the event that imported models are changed.</p></td><td>Yes</td></tr><tr><td>name</td><td>Optionally, a name for the field that overrides the referenced field's name. Otherwise, the field name defaults to that of the referenced field.</td><td>No</td></tr><tr><td>jsonName</td><td>Optionally overrides the referenced field's jsonName.</td><td>No</td></tr><tr><td>id</td><td><p>Optionally overrides the id specified by the field being referenced.</p><p><em>Field ids must be between 0 and 32767 inclusive</em></p></td><td>No</td></tr><tr><td>required</td><td>Whether or not the field is a required field for the type. Fields declared as required will cause a check to be added for the value being set in the generated types isValid() method.</td><td>No</td></tr><tr><td>isKey</td><td>True if this field should serve as the key when inserting the entity into a LongMap or StringMap collection. In this case, insertion of the message in the collection with a given key will update this field with the key value.<br><br><em>Note: Only applies to state entity elements</em></td><td>No</td></tr><tr><td>doc</td><td>The doc to use for the field which overrides that specified by the referenced field.</td><td>No</td></tr><tr><td>deprecated</td><td>When true all methods generated for the field will be marked as deprecated.</td><td>No</td></tr></tbody></table>

{% hint style="warning" %}
**Qualifying conflicting type names**

It is not possible to define two types with the same name in a given model. One exception to this rule is that it is possible (though strongly discouraged) to define a type with the same name as a built-in type. In this case, using an unqualified type reference will favor the built-in type. In this case, it is possible to reference the local type by using the *this* keyword, or the fully qualified name:

```
<model xmlns="http://www.neeveresearch.com/schema/x-adml"
       xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"
       name="Trading" 
       namespace="com.mycompany.trading" 
       defaultFactoryId="100">
       doc="Contains Messages used by the Trading Application">
 
  <import model="../orderprocessing.xml"/>
 
  <!--An unfortunately named entity:-->
  <entity name="Currency" id="1">
    <field name="currencyCode" type="String"/>
  </entity>
 
  <entity name="MyEntity" id="2">
    <!--
      Without qualifying the Currency type, the built
      in Currency type is used, and the generated entity
      will return a java.util.Currency.
    -->
    <field name="javaCurrency" type="Currency" />
    <!-- 
      The 'this' keyword indicates that this is the Currency
      entity defined in this model:
    -->
    <field name="currencyEntity" type="this.Currency"/>
    <!--
      Namespace qualifying the typeindicates that this is the Currency
      entity defined in this model:
    -->
    <field name="currencyEntity2" type="com.mycompany.trading.Currency"/>
    <!--
      Namespace qualifying the type indicates that this is the Currency
      entity defined in some imported model:
    -->
    <field name="currencyEntity3" type="com.mycompany.orderprocessing.Currency"/>
  </entity>
</model>
```

{% endhint %}

### Semantic Types

The ADM model provides a section that can be used to define types based on the platform's primitive and built-in types. Such semantic types can be used in place of their corresponding primitive type in the model and will inherit their documentation. The type used for the field in generated messages and entities will be the base type specified by the named field.

#### ADM Model

```xml
<!-- Defines semantic types that can be referenced elsewhere in the model in place -->
<types>
    <type name="SocialSecurityNumber" base="String" length="12" poolable="true">
            doc="Represents a social security number in the format xxx-xx-xxxx">
            <documentation>
          <![CDATA[
            Detailed information about social security numbers can be found at the <a href="http://www.ssa.gov/">Social Security Website</a>.
          ]]>
        </documentation>
    </type>
    <type name="Salary" base="float" doc="Represents a salary in US dollars."/>
</types>
 
<fields>
    <field name="ssn" type="SocialSecurityNumber" id="1"/>
    <field name="salary" type="Salary" id="2"/>
</fields>
 
<messages>
    <message name="EmployeeSalarayUpdateMessage" factoryid="1" id="1">
        <fieldRef ref="ssn" required="true" />
        <fieldRef ref="salary" required="true" />
    </message>
</messages>
```

#### Generated Code

Note that "Price" doesn't result in a new java type being created. The generated source uses its base type (float) directly and inherits the documentation of the semantic type. This is true of all semantic types, with the exception of types with a base of String that are declared as poolable (see [Poolable String Types](#poolable-string-types) below).

```java
public interface IEmployeeSalarayUpdateMessage extends IRogNode, IRogMessage {
    /**
     * Sets the value of this field to the provided SocialSecurityNumber value.
     */
    public void setSsn(final String val);
 
    /**
     * Gets the value of 'ssn'
     *
     * @return The value of 'ssn'
     */
    public String getSsn(); 
     
    /**
     * Sets the value of 'salary'
     * <p>     
     * <h2>Field Semantics</h2>     
     * <p>
     * Represents a salary in US dollars.
     *
     * @param val The value to set.
     */
    public void setSalary(final float val);
    /**
     * Gets the value of 'salary'
     * <p>     
     * <h2>Field Semantics</h2>
     * <p>
     * Represents a salary in US dollars.
     *
     * @return The value of 'salary'
     */
    public float getSalary();
}
 
/**
 * <p>
 * Represents a social security number in the format xxx-xx-xxxx.
 * <p>
 * Detailed information about social security numbers can be found at the <a href="http://www.ssa.gov/">Social Security Website</a>.
 */
final public class SocialSecurityNumber extends XAbstractPooledString<SocialSecurityNumber> {

    /** The default factory used to create new SocialSecurityNumber instances. */
    final public static Factory FACTORY;

    /** An optionally pool backed factory for SocialSecurityNumber. */
    public static class Factory extends XAbstractPooledString.PooledStringFactory<SocialSecurityNumber> {...}

    ...
}
```

See [Poolable String Types](#poolable-string-types) below for what the generated type provides and how to use it.

### Poolable String Types

A String semantic type or field declared `poolable="true"` is the one case where the code generator creates a new Java type rather than using the base type directly. For those, it generates a subclass of `XString` for the field, along with the factory that allows instances of it to be pooled and preallocated.

The motivation is not garbage alone. `XString` already provides zero garbage manipulation of string values, which is why low latency applications hold `XString` rather than `java.lang.String` in domain state and copy values in and out of messages instead of allocating new ones. What that does not avoid is promotion of those objects across heap generations, which is expensive. Avoiding promotion requires pooling and preallocating the strings held in application state, and a poolable type is what makes that possible.

For the runtime patterns, including preallocation, see [XStrings](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/xstrings).

#### Declaring a String Field as Poolable

A poolable type is generated for a String field in any of three cases:

* The field element declaring it carries `poolable="true"`. The generated type is named after the field.
* A semantic type is declared with a base of `String` and `poolable="true"`, as `SocialSecurityNumber` is above. The generated type is named after the semantic type.
* The code generator is run with the `generateAllStringsPoolable` directive, which makes every String field in the model generate a poolable type regardless of its own `poolable` attribute. This suits applications that would rather keep the notion of pooling out of the model itself.

The `length` attribute sizes the string's backing buffer.

**Restriction.** A generated poolable string type may not collide with the name of a built-in or primitive type (`string`, `integer`, `Currency` and so on), with a type in `java.lang`, or with any other type declared in the model, including enumerations, entities, messages and collections. Where a field name would produce such a collision, declare a semantic type with a different name and use that for the field instead.

#### Generated Code

For the `SocialSecurityNumber` semantic type declared above, the generator produces the type, a nested `Factory`, and a default `FACTORY` instance:

```java
final public class SocialSecurityNumber extends XAbstractPooledString<SocialSecurityNumber> {

    /** The default factory used to create new SocialSecurityNumber instances. */
    final public static Factory FACTORY;

    /** An optionally pool backed factory for SocialSecurityNumber. */
    public static class Factory extends XAbstractPooledString.PooledStringFactory<SocialSecurityNumber> {

        protected Factory(String name,
                          int stringLength,
                          boolean isNative,
                          boolean pooled,
                          int preallocationCount,
                          boolean threaded) {...}

        @Override
        protected final int getDefaultInitialStringLength() {...}
    }

    /**
     * Constructs a new Factory for SocialSecurityNumber.
     *
     * @param name A name unique to this factory type which is used to uniquely identify the underlying pool.
     * @param stringLength The expected length of the string which is used to size its backing buffer.
     * @param pooled Whether or not to back the factory with a pool.
     * @param preallocationCount The number of instances to preallocate.
     * @param threaded Whether or not the backing pool should be thread safe.
     * @param isNative Whether or not the strings should use native backing buffers (when enabled/supported).
     */
    public static Factory newFactory(final String name,
                                     final int stringLength,
                                     final boolean pooled,
                                     final int preallocationCount,
                                     final boolean threaded,
                                     final boolean isNative) {...}
}
```

Fields of a poolable type are given accessors that copy the value in and out without allocating:

```java
public interface IEmployeeSalarayUpdateMessage extends IRogNode, IRogMessage {

    /** Copies the value of this String field into a new XString from the given factory. */
    public <T extends XString> T getSsnTo(final XString.Factory<T> factory);

    /** Sets the value of this field from an already encoded XString. */
    public void setSsnFrom(final XString val);
}
```

`getSsnTo` is a zero garbage operation when the factory it is given is backed by a pool with instances available, which is what makes the factory worth configuring.

#### Pooled String Type Name Conflicts

Types generated from a *field* name are more prone to collision than those generated from a semantic type, because field names are chosen for readability rather than uniqueness. Two code generator directives make such collisions easier to manage:

* `pooledStringFieldTypeNameSuffixPolicy` sets the policy for resolving them.
* `pooledStringFieldTypeNameSuffix` sets the suffix applied, defaulting to `String`.

The policies are:

| Policy           | Behaviour                                                                                                                                                                                                                                   |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `None` (default) | Field names are not suffixed. A conflict results in a code generation or compilation error. Useful when you would rather be told about a conflict and resolve it deliberately, typically by declaring a semantic type with a distinct name. |
| `Always`         | The generated type is always suffixed, which makes a conflict unlikely.                                                                                                                                                                     |
| `OnConflict`     | The suffix is applied only when the generated name would conflict with another type.                                                                                                                                                        |

These directives are covered with the rest of the generator configuration under [The Code Generator](/talon/developing-applications/modeling-messages-and-state/the-code-generator).

### Entities

Entities are defined in the entities element of the model and must have a unique id with respect to other types within the scope of their factory.

#### ADM Model

```xml
<entities>
  <entity name="EntityA" factoryid="1" id="1">
    <field name="enumField" type="Enumeration" doc="tests an enum field" id="1"/>
    <field name="enumArrayField" type="Enumeration[]" doc="tests an enum array field." id="2"/>
    <field name="intEnumField" type="IntEnumeration" id="3"/>
    <field name="intEnumArrayField" type="IntEnumeration[]" id="4"/>
    <field name="charEnumField" type="CharEnumeration" id="5"/>
    <field name="charEnumArrayField" type="CharEnumeration[]" id="6"/>
    <field name="stringEnumField" type="StringEnumeration" id="7"/>
    <field name="stringEnumArrayField" type="StringEnumeration[]" id="8"/>
    <field name="booleanField" type="Boolean" id="9"/>
    <field name="booleanArrayField" type="Boolean[]" id="10"/>
    <field name="byteField" type="Byte" id="11"/>
    <field name="byteArrayField" type="Byte[]" id="12"/>
    <field name="shortField" type="Short" id="13"/>
    <field name="shortArrayField" type="Short[]" id="14"/>
    <field name="intField" type="Integer" id="15"/>
    <field name="intArrayField" type="Integer[]" id="16"/>
    <field name="longField" type="Long" id="17"/>
    <field name="longArrayField" type="Long[]" id="18"/>
    <field name="floatField" type="Float" id="19"/>
    <field name="floatArrayField" type="Float[]" id="20"/>
    <field name="doubleField" type="Double" id="21"/>
    <field name="doubleArrayField" type="Double[]" id="22"/>
    <field name="stringField" type="String" id="23"/>
    <field name="stringArrayField" type="String[]" id="24"/>
    <field name="dateField" type="Date" id="25"/>
    <field name="dateArrayField" type="Date[]" id="26"/>
    <field name="charField" type="Char" id="27"/>
    <field name="charArrayField" type="Char[]" id="28"/>
    <field name="currencyField" type="Currency" id="29"/>
    <field name="currencyArrayField" type="Currency[]" id="30"/>
    <field name="embeddedEntityField" type="EmbeddedEntity" id="32"/>
    <field name="embeddedEntityArrayField" type="EmbeddedEntity[]" id="33"/>
    <field name="entityField" type="EntityB" id="34"/>
    <field name="entityBMapField" type="EntityBLongMap" id="35"/>
  </entity>
  
  </entity name="EntityB" factoryid="1" id="2">
    <field name="intField", type="Integer" id="1"/>
  <entity>
  
  <entity name="EmbeddedEntity" factoryid="1" id="3" asEmbedded="true">
    <field name="longField", type="Long" id="1"/>
  </entity>
    
</entities>
```

**Entity Element Attributes**

<table><thead><tr><th width="152">Attribute</th><th width="496">Description</th><th>Required</th></tr></thead><tbody><tr><td>name</td><td>The name of the field, must be unique within the model.</td><td>Yes</td></tr><tr><td>id</td><td><p>The id of the entity, which must be unique within the scope of all types in the entity's factory.</p><p><em>Entity / Message ids must be between 0 and 32767 inclusive</em></p></td><td>Yes</td></tr><tr><td>factoryId</td><td>The id of the factory, which must be unique within the scope of all factories used within an application. When a message is received, the factoryId and entityId uniquely identify the type to be deserialized. factory IDs &#x3C;= 0 are reserved for platform use.</td><td>Yes</td></tr><tr><td>asEmbedded</td><td>Defaults to false, Indicates whether or not this entity is generated to be used as an embedded or child field of another entity. Embedded entities are always serialized transported with its parent entity. Entities used as fields in messages must be declared as embedded.</td><td>No</td></tr><tr><td>transactional</td><td>Whether or not this entity supports transaction commit and rollback via the applications ODS store.</td><td>No</td></tr></tbody></table>

#### Entity Field Element Attributes

<table data-header-hidden><thead><tr><th width="151">Attribute</th><th width="499">Description</th><th>Required</th></tr></thead><tbody><tr><td>type</td><td><p>The type of the element.</p><p>If the type is defined in this namespace, is defined in only one of the imported models, or is a primitive or collection type, only the simple name of the type need be used. A non-fully-qualified name will be looked up in both the current model and its imports. If name is declared in the current model and in one of the imports, the type from the current model will take precedence. If multiple imports define the same name, an error will be reported. The best correction in such case is to use the fully-qualified name.</p><p>If the field is an array type, it should be suffixed with array indices such as MyEntity[] to denote it as an array.</p></td><td>Yes</td></tr><tr><td>name</td><td>The name of the field, must be unique within the model.</td><td>Yes</td></tr><tr><td>jsonName</td><td>Contains the name of the JSON property that will be used for the field when the message is serialized to JSON. Defaults to using the value defined in name.</td><td>No</td></tr><tr><td>id</td><td><p>The id of the field. The id must be unique with the scope of this entity. For Xbuf/Protobuf encoding, this tag is used as the tag value for the field on the wire. If not set, a unique id will be generated by the source code generator. For better control over compatibility, it is recommended that application set this value manually.</p><p><em>Field ids must be between 0 and 32767 inclusive</em></p></td><td>No</td></tr><tr><td>isKey</td><td>True if this field should serve as the key when inserting the entity into a LongMap or StringMap collection. In this case, insertion of the message in the collection with a given key will update this field with the key value.</td><td>No</td></tr></tbody></table>

#### Generated Source

For an entity named "MyEntity" an interface and an implementation will be generated using the model's namespace. Entities will extend IRogNode or IRogContainerNode marking that they can be used as nodes with ODS.

```java
@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IEntityA extends IRogContainerNode {...}
```

```java
@Generated(value="com.neeve.adm.AdmXbufGenerator", date="Fri Jan 23 02:03:22 PST 2015")
final public class EntityA extends RogContainerNode implements IEntityA, IXbufDesyncer, IRogJsonizable {...}
```

### Messages

Messages are defined in the messages element and must have a unique id with respect to other types within the scope of their factory. Messages can use all modeling capabilities of entities, but cannot use non-embedded entities or non-array collections as fields.

#### ADM Model

```xml
<messages>
  <message name="SampleMessage" factoryid="1" id="1">
    <field name="sno" type="Long" id="10000"/>
    <field name="symbol" type="String" id="10001" />
    <field name="price" type="Float" id="10002"/>
    <field name="embeddedEntity" type="EmbeddedEntity" id="10003"/>
    <field name="embeddedEntityArray" type="EmbeddedEntity[]" id="10003"/>
  </message>
</messages>
```

#### **Message Attributes**

Messages support the same attributes as entities (listed above) with the exception that transactional defaults to false for messages.

#### **Message Field Attributes**

Message fields support the same attributes as entities (listed above).

#### **Notes**

For Xbuf and Protobuf encoding, the id is used to generate the protobuf field tag, so changing field ids will break wire compatibility. If fields are not explicitly assigned IDs in the model, then the ADM generate will assign them automatically. in this case, fields should not be removed or changed in order to maintain wire compatibility with earlier versions of the generated code.

Messages can only declare primitive types, built-in types, embedded types as fields (or arrays of those types). Collections, Messages, and non-embedded Entities can't be used.

#### Generated Source

For an entity named "MyMessage" an interface and an implementation will be generated using the model's namespace. Entities will extend IRogNode or IRogMessage, marking that they can be used as nodes with the platform's ODS and SMA modules.

```java
@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IMyMessage extends extends IRogNode, IRogMessage {...}
```

```java
@Generated(value="com.neeve.adm.AdmXbufGenerator", date="Fri Jan 23 02:03:22 PST 2015")
final public class MyMessage extends RogNode implements IMessage, ILnkMessage, MessageReflector, IXbufDesyncer, IRogJsonizable  {...}
```

### Collections

Collections are defined in the element and must also have a unique id with respect to other types within the scope of their factory. Collections may not be declared as embedded at this time.

#### ADM Model

```xml
<collections>
  <collection name="MyQueue" is="Queue" contains="MyEntity"
    factoryid="1" id="5" />
  <collection name="MyLongMap" contains="MyEntity" is="LongMap"
    factoryid="1" id="6" />
</collections> 
<entity name="MyEntity" factoryid="1" id="1">
 <field name="longKey" type="Long" isKey="true" id="1"/>
 <field name="aQueue" type="MyQueue" id="2"/>
 <field name="aMap" type="MyStringMap id="3"/>
</entity>
```

#### Generated Source

```java
@Generated(value="com.neeve.adm.AdmGenerator", date="Fri Jan 23 02:03:22 PST 2015")
public interface IChildLongMap extends IRogLongMap<Child1>, Map<Long, Child1> {}
 
@Generated(value="com.neeve.adm.AdmXbufGenerator", date="Fri Jan 23 02:03:22 PST 2015")
final public class ChildLongMap extends RogLongMap<Child1> implements IChildLongMap, IXbufDesyncer, IRogJsonizable {...}
```


# The Code Generator

The `com.neeve.tools.AdmCodeGenerator` class, included in Talon ADM module (`nvx-rumi-adm-<version.jar`), is the main code generator that facilitates code generation. The `AdmCodeGenerator` creates classes from a model, organizing them into a namespace-qualified directory structure based on a specified root directory. Optional parameters can be provided.

Talon also offers plugin for build tools, such as Maven, to integrate code generation with the build lifecycle of Maven based applications.

## Encoding Types

All classes created by the code generator ensure efficient serialization for storage and network transfer. Talon supports various serialization formats, known as encoding types. The code generator suppirts a parameter to specify the desired encoding type for which the classes are to be generated.

The following are the encoding types currently supported by the ADM code generator:

* **Json**: Javascript Object Notation serialization; not very efficient but highly readable
* **Protobuf**: Google's protobuf serialization format, which can be useful for portability purposes providing a reasonable balance between performance and interoperability. The generated classes for this encoding type internally use the classes generated by the Google Protobuf IDL compiler.
* **Xbuf2**: Talon's protobuf implementation. The classes generated by this encoding type support zero garbage operation, do not use the Google Protobuf IDL compiler and are 100% wire compatible with Google's protobuf wire format.

{% hint style="info" %}
**Deprecated Encoding Types**

The `Xbuf` encoding format, which was a precursor to the `Xbuf2` encoding format has been deprecated. Users of the `Xbuf` encoding format should migrate to the `Xbuf2` encoding format. See [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type) for more information.
{% endhint %}

{% hint style="warning" %}
**Encoding Types Scheduled For Deprecation**

Going forward, classes generated for all ADM encoding types will support serialization to and deserialization from the JSON serialized format. Therefore, the JSON encoding format will no longer be available starting with the next major X Platform release.

Starting with the next major release, The `Xbuf2` encoding type will fully replace the `Protobuf` encoding type i.e. there will no capability to generate classes that produce and use classes generated by the Google Protobuf IDL compiler.
{% endhint %}

See [Choosing an Encoding Type](/talon/developing-applications/modeling-messages-and-state/the-code-generator/choosing-an-encoding-type) for more information on choosing an appropriate encoding.

### Mixing Encoding Types

You can mix different encoding types across models at runtime, provided each model has a unique factory id ((since the factory id is used to determine which generated factory to use to decode a message). However, it is not legal to mix and match encodings within a model via import. For example, if you generate one model with `Protobuf`, it is not possible to use types generated using that model in a model generated with `Xbuf2`.

## Schema Location

The XML schema for the ADM language (for the input models) - `xadml.xsd` - can be found at the root of `nvx-rumi-adm-<version>.jar`

## Generated Class Namespace

Each model is associated with a namespace. This namespace is the default package for generated classes from a code generator. Users can specify a different namespace when invoking the code generator, allowing generation of sources with varied encodings of the same model by rerunning the generator with distinct namespaces.

## Imported Model Resolution

To handle multi-module and multi-project builds, the ADM XML Parser, and Maven Build plugin search for model imports on the classpath using the imported model's fully qualified namespace. The model xml (and .proto for Xbuf2/Protobuf) will be copied to the generated source and target classes folder in fully qualified form for inclusion in the project's jar.

Import resolution searches both current classpath and OS file system when resolving an import. Recommended is to use relative classpath strings whenever possible. Consider the following example project with two models, `model.xml` imports `other_model.xml`:

```
${basedir}/src/some/package/model.xml [In xml we define namespace = "some.model.namespace"]
${basedir}/src/other/package/other_model.xml [In xml we define namespace = "some.other_model.namespace"]
```

The recommended way to define import would be one of the following:

{% code fullWidth="false" %}

```xml
Import Through Classpath

<!--
Method #1: (recommended) Import from relative model namespace.
Imports from classpath relative to namespace of importing model,
in this case some.model.namespace.
This would try resolving from
some/model/namespace/../../other_model/namespace/other_model.xml.
Evaluating ../.. would give us resource some/other_model/namespace/other_model.xml
-->
<import model="../../other_model/namespace/other_model.xml" />
 
 
<!-- Method #2 (recommended): Import from full model namespace.
This will work if generated sources are compiled and added to classpath.
For example if we have project dependencies where one project imports a
model XML from another project, the original XML file is not available,
but the one in the produced jar is. Therefore, we are getting file from
classpath, where project dependency containing import is added to the classpath.
For maven this also works within the same project if code generation for imported
other_model.xml is run first. So, when we reach in pom the code generation of model.xml,
other_model.xml is already generated and xml is copied to
target/classes/some/other_model/namespace/other_model.xml.
target/classes is already in classpath so this will resolve successfully to other_model.xml-->
<import model="some/other_model/namespace/other_model.xml" />
```

{% endcode %}

The reason why this works is that the result of running code generation would be:

```
${basedir}/target/classes/some/model/namespace/model.xml
${basedir}/target/classes/some/model/namespace/Model.proto
${basedir}/target/classes/some/model/namespace/GeneratedModelClass1.class
${basedir}/target/classes/some/model/namespace/GeneratedModelClass2.class
...
${basedir}/target/classes/some/other_model/namespace/other_model.xml
${basedir}/target/classes/some/other_model/namespace/OtherModel.proto
${basedir}/target/classes/some/other_model/namespace/GeneratedOtherModelClass1.class
${basedir}/target/classes/some/other_model/namespace/GeneratedOtherModelClass2.class
... 
```

The requirement is that code generation first runs for other\_model.xml so that it can be found in classes folder at the time of running model.xml.

The following modes of import model resolution are provided for advanced use cases but are discouraged.

```xml
From Project Source Folder Through OS Filesystem

<!-- Method #3: Import from absolute filesystem path: -->
<import model="absolute path to file other_model.xml in the OS filesystem" />
 
<!-- Method #4:
Import as path relative to the provided modelsDir provided to the code generator
${modelsDir}/other/package/other_model.xml.
-->
<import model="other/package/other_model.xml" />
<!-- Method #5:
Import as relative filesystem path. Path is relative to
model that does the import so this is evaluated to
${basedir}/src/some/package/../other/package/other_model.xml.
If we evaluate ../ we get ${basedir}/src/some/other/package/other_model.xml
-->
<import model="../other/package/other_model.xml" />
```

## Generated Protobuf IDLs

The `Xbuf2` and `Protobuf` code generators generate the following Protobof IDLs (.proto file).

* Main IDL
  * This IDL is generated from the contents of the input model
* Platform Bundled IDLs
  * descriptor.proto
  * AdmTypes.proto

### Main IDL

The Main IDL is generated for use by the user as well as for internal use by the `Protobuf` code generator. The `Xbuf2` code generator generates it solely for the user and does not use it internally.

The Main IDL is output in the generated source folder using the model's fully qualified namespace.

### Platform Bundled IDLs

The code generator bundles the following IDLs in the packaged jars to support custom options used by ADM to enrich handling of enums, and to define additional data types used by ADM.

<table data-header-hidden><thead><tr><th width="178">IDL</th><th width="402">Full Path</th><th>Description</th></tr></thead><tbody><tr><td>descriptor.proto</td><td>google/protobuf/descriptor.proto</td><td>Defines options available in Protobuf</td></tr><tr><td>AdmTypes.proto</td><td>com/neeve/adm/types/protobuf/AdmTypes.proto</td><td>Defines custom enum options and additional ADM data types</td></tr></tbody></table>

## Build Tool Integration

Talon currently supports the use of following code generators

* `com.neeve.tools.AdmCodeGenerator`
* Maven Plugins (layered on the AdmCodeGenerator)

The following describes how these can be run with various build tools.

### **Java**

```bash
java –cp nvx-rumi-kernel-<version>.jar:nvx-rumi-core-<version>.jar:nvx-rumi-io-<version>.jar:nvx-rumi-adm-<version>.jar:[:jars-containing-imports] com.neeve.tools.AdmCodeGenerator -e <Encoding Type> –f <model-file>.xml -o target/generated-sources/nvx-adm
```

### **ANT**

```xml
<target name="run-adm-generator">
    <java classname="com.neeve.tools.AdmCodeGenerator" 
          fork="true" 
          failonerror="true" 
          jvm="${build.java.home}/bin/java" 
          classpathref="build.classpath">
        <arg value="-e"/>
        <arg value="Xbuf"/>
        <arg value="-f"/>
        <arg value="${basedir}/src/main/java/com/acme/messages/messages.xml"/>
        <arg value="-o"/>
        <arg value="-o target/generated-sources/nvx-adm"/>
        <arg value="-d generateEmbeddedEntityInterfaces=false"/>
    </java>
</target>
```

### Maven

{% hint style="info" %}
**Parallel Build Support**: The ADM Maven Plugin is thread-safe and supports parallel Maven builds (`mvn -T <threads>`). This can significantly reduce build times in multi-module projects that use the ADM code generator.
{% endhint %}

When working with Maven, the following are the various options on how one can use the ADM code generator

* Using the Maven Exec Plugin
* Using the Maven X Platform Plugins
  * The ADM Plugin
  * The Platform Plugin

#### **Using the Maven Exec Plugin**

```xml
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>1.3.2</version>
    <executions>
        <execution>
            <id>generate-messages</id>
            <goals>
                <goal>exec</goal>
            </goals>
            <phase>generate-sources</phase>
            <configuration>
                <executable>java</executable>
                <arguments>
                    <argument>-classpath</argument>
                    <classpath/>
                    <argument>com.neeve.tools.AdmCodeGenerator</argument>
                    <argument>-e</argument>
                    <argument>Xbuf2</argument>
                    <argument>-o</argument>
                    <argument>${project.build.directory}/generated-sources/myapp/messages</argument>
                    <argument>-f</argument>
                    <argument>${project.basedir}/src/main/models/com/mycompany/myapp/messages/messages.xml</argument>
                </arguments>
            </configuration>
        </execution>
    </executions>
</plugin>
```

#### **Using the Maven ADM Plugin**

<table><thead><tr><th width="286">Plugin Goal</th><th>Description</th></tr></thead><tbody><tr><td>generate, adm-generate</td><td>Generates code to a generated sources folder which would be included in the built jar.</td></tr><tr><td>generateTest, adm-generateTest</td><td>Generates code to a generated test sources folder which would be included in the built test jar</td></tr></tbody></table>

```xml
<plugins>
  <plugin>
    <groupId>com.neeve</groupId>
    <artifactId>nvx-adm-maven-plugin</artifactId>
    <version>${nvx.adm.version}</version>
    <executions>
      <execution>
        <id>Messages</id>
        <phase>generate-sources</phase>
        <goals>
        <goal>generate</goal>
        </goals>
        <configuration>
          <modelFile>${basedir}/src/main/java/com/mycompany/messages/messages.xml</modelFile>
          <encodingType>Xbuf2</encodingType>
          <directives>
            <generateEmbeddedEntityInterfaces>false</generateEmbeddedEntityInterfaces>
          </directives>
        </configuration>
      </execution>
     </executions>
   </plugin>
</plugins>

```

#### **Using the Maven Platform Plugin**

If your project is using the `nvx-platform-bom` and you want to generate code with the same version of the platform you are using, you may also use the `nvx-platform-maven-plugin`. The advantage of this approach is that you can use the same version of the plugin as the platform bom. Because maven BOMs don't cover plugin versions, using the `nvx-adm-maven-plugin` would mean that the `nvx-adm-maven-plugin` version would have to be specified separately.

```xml
<plugins>
  <plugin>
    <groupId>com.neeve</groupId>
    <artifactId>nvx-platform-maven-plugin</artifactId>
    <version>${nvx.platform.version}</version>
  </plugin>
</plugins>
```

#### Gradle, Ivy & Others

To create classes using the ADM code generator with build tools like Gradle, Ivy & others, invoke `com.neeve.tools.AdmCodeGenerator` using the appropriate mechanisms offered by these tools.

## Code Generator Options

The following options are available for the ADM code generation:

<table><thead><tr><th width="213">Command Line</th><th width="304">Maven Plugins (ADM &#x26; Platform)</th><th width="272">Description</th><th>Default</th></tr></thead><tbody><tr><td>-f, --file</td><td>modelFile</td><td>The input file specified either as OS path or URL</td><td>-</td></tr><tr><td>-m, --modelsdir</td><td>modelsDirectory</td><td>Base directory to use when searching for imported models that are not found on the project / generator's classpath.</td><td>-</td></tr><tr><td>-o, --outdir</td><td>projectOutputDirectory</td><td><p>Base output directory for the generated files.</p><p>For maven plugin defaults to target/generated-[test]-sources/nvx-adm</p></td><td>-</td></tr><tr><td>-c, --classesdir</td><td>classesOutputDirectory</td><td>Classes output folder (to which generated resoures should be copied). May be specified multiple times to copy to multiple directories.</td><td>-</td></tr><tr><td>-e, --encoding</td><td>encodingType</td><td>Encoding type of content underlying the generated classes (Xbuf | Protobuf | Json)</td><td>Protobuf</td></tr><tr><td>-x, --xpcompat</td><td>protoXbufGenerationCompatibility</td><td>Wire compatibility between protobuf and xbuf generated classes (None | Xbuf | Protobuf</td><td>Protobuf</td></tr><tr><td>-y, --emptyifnullarray</td><td>generateArrayGetterEmptyIfNull</td><td>Instructs the code generator return empty arrays instead of null for unset array fields</td><td>false</td></tr><tr><td>-n, --namespace</td><td>namespace</td><td>Namespace override of model parsed from the input file (overrides namespace in model file if supplied)</td><td>-</td></tr><tr><td>-p, --protodir</td><td>N/A</td><td>An additonal directory in which to search for imported .proto files.</td><td>-</td></tr><tr><td>-d, --directive</td><td>directives</td><td><p>A key=value pair specifying a code generation directive. (May be specified multiple times).</p><p>See <a href="#code-generator-directives">Directives </a>below.</p></td><td>-</td></tr><tr><td>-b, --buildinfo</td><td>buildInfo</td><td>String with build-time information such as project version, timestamp or machine. This is added to the AdmGenerated annotation of generated classes.</td><td>-</td></tr><tr><td>-i, --incremental</td><td>incrementalBuild</td><td>Trigger incremental code generation - run only if something changed since last run</td><td><p>false</p><p>(true for maven plugin)</p></td></tr><tr><td>N/A</td><td>useBasicDeltaDetection</td><td>When running an incremental build, basic delta detection indicates that model's are rebuilt based on whether source model timestamp. With advanced delta detection dependencies are examined as well.</td><td>false</td></tr><tr><td>-u, --bundledir</td><td>modelBundleOutputDirectory</td><td>Directory to which to output model XML and IDL files if applicable</td><td>-</td></tr><tr><td>N/A</td><td>generateModelBundle</td><td>Indicates whether models with all their dependencies and IDLs should be output to modelBundleOutputDirectory</td><td>false</td></tr><tr><td>N/A</td><td>codegenListenerClassName</td><td>Class name of external listener to receive events from code generator. See <a href="#generatingsourcecode-incrementalcoderegeneration">ADM Code Generation Events</a>.</td><td>-</td></tr><tr><td>N/A</td><td>codegenListenerProperties</td><td>Additional properties to pass to the constructed code gen listener.</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td></tr></tbody></table>

## Code Generator Directives

Some advanced properties can be passed to the code generator as directives. The following is a list of supported directives:

<table><thead><tr><th width="354">Directive</th><th width="379">Description</th><th>Default</th></tr></thead><tbody><tr><td>requireExplicitCollectionKeys</td><td><p>Directive indicating that map collections' contained entities must define an explicit field field to store the key for the entity when it is in the map. When this directive is false, it is possible that as the model evolves the implicitly generated key field could change and cause existing keys to be ignored on upgrade.</p><p>💡 It is recommend that new projects set this directive to true.</p><p>SINCE 3.11</p></td><td>false</td></tr><tr><td>generateEmbeddedEntityInterfaces</td><td>Directive indicating that the generator should create interfaces for embedded entities. This can be disabled for applications with stringent performance requirements to reduce the overhead associated with multi-morphic vtable lookups.</td><td>true</td></tr><tr><td>generateEmbeddedEntitiesNonFinal</td><td><p>When this directive is set to true the generated entity class is not declared as final nor are its accessors. This feature can be useful for applications that need to mock embedded entities in test frameworks such as CGLIB, but is not recommended for production use for performance reasons.</p><p>SINCE 3.8.189</p></td><td>false</td></tr><tr><td>generateDefaultGetters</td><td>Whether or not to generate default getters that accept a value to return when the field is not set. <em>Not typically recommended</em></td><td>false</td></tr><tr><td>generateThrowOnUnsetGetters</td><td>Whether or not to generate getXXXOrThrow() or accessors that will throw an ERogFieldNotSetException when the field has not been set. This provides an alternative to calling hasXXX for a field to test if the field is unset. <em>Usage of this directive is not recommended; hasXXX is the recommended approach to testing if a field is not set. Exception throwing is more expensive, and the generated getXXXOrThrow method introduces extra invocation overhead and a larger code size.</em></td><td>false</td></tr><tr><td>generateRequiredFieldValidators</td><td>Whether or not validation logic is generated in the types validators for required fields. Enabling this leads large generated code size, and validation checks are expensive, so this is not recommended for performance sensitive applications.</td><td>false</td></tr><tr><td>generateFluentSetters</td><td>Directive indicating that fluent style setters should be generated for fields. This can be enabled to generate fluent accessors on generated types. This can be useful for writing concise test code, but is more overhead, so it's usage is not typically recommended.</td><td>false</td></tr><tr><td>generateAllStringsPoolable</td><td>Directive indicating that all Strings fields in the model should be generated as poolable types regardless of the value of the field's poolable attribute.</td><td>false</td></tr><tr><td>pooledStringFieldTypeNameSuffixPolicy</td><td>Can specify None, Always or OnConflict to instruct the code generator as to how to handle naming conflicts that arise from a pooled string field type name generated from a field name are suffixed to avoid a name clash.</td><td>"None"</td></tr><tr><td>pooledStringFieldTypeNameSuffix</td><td>Specifies the suffice to use to resolve pooled string type name conflicts with Always or OnConflict suffixing policies.</td><td>"String"</td></tr><tr><td>generateProtobufClasses</td><td>Specified that protobuf classes should be generated using the protoc code generator in addition to the encoding type specific generated classes. This directive only applies to Xbuf and Xbuf2 encoding types.</td><td>"String"</td></tr></tbody></table>

## Code Generation Events <a href="#generatingsourcecode-incrementalcoderegeneration" id="generatingsourcecode-incrementalcoderegeneration"></a>

ADM code generation is run by using `com.neeve.tools.AdmCodeGenerator` class. It is possible to supply a listener to the `AdmCodeGenerator` instance to subscribe to events that are fired at certain points in code generation run. The events are in the `AdmCodeGenerator.CodeGenerateEventType` enumeration as follows.

```java
Event Types
/**
 * Code generation steps that external listener can listen for.
 */
public enum CodeGenerateEventType {
    /**
     * When code generation is about to run.
     */
    START,
    /**
     * After model has been parsed
     */
    MODEL_PARSED,
    /**
     * When code generation finished.
     */
    END,
    /**
     * When code generation is skipped because nothing changed since previous run
     * Can happen when incrementalBuild is on.
     */
    SKIP
}
```

An event listener can be supplied via the `CODEGEN_EVENT_LISTENERS` parameter to an instance of `AdmCodeGenerator`. The Listener interface is defined in `AdmCodeGenerator` as follows:

```java
**
 * Event listener for code generation events.
 */
public interface CodegenListener {
    /**
     * When code generation event is triggered this method will be called.
     */
    void codeGenerateEvent(CodeGenerateEvent e);
}
```

The Maven plugin exposes this capability via the `codegenListenerClassName` configuration parameter. This parameter accepts the fully qualified name of the class that implements the listener interface. While the `AdmCodeGenerator` can accept multiple listeners, the Maven plugin accepts only one class and will create only one instance of that listener class per execution. The class shpuld be in project's build classpath (either in project being built or in one of its dependencies).

Each event is dispatched with an instance of `AdmCodeGenerator.CodeGenerateEvent` that contains the data for the event. The following are the key methods in this class

```java
/**
 * Holds event data for code generation events.
 */
public class CodeGenerateEvent {
    /**
     * @return the eventType
     */
    public final CodeGenerateEventType getEventType() {
        return eventType;
    }
    /**
     * @return the model. Only available if eventType is {@link CodeGenerateEventType#MODEL_PARSED} or {@link CodeGenerateEventType#END}.
     */
    public final AdmModel getModel() {
        return model;
    }
    /**
     * @return the errorAggregator. Only available if eventType is {@link CodeGenerateEventType#MODEL_PARSED} or {@link CodeGenerateEventType#END}.
     */
    public final AdmSourceCodeErrorAggregator getErrorAggregator() {
        return errorAggregator;
    }
}
```

### Sample: Running Additional Model Validations

Developers may tap into listener mechanism to perform additional model validations as given in example below. The example demonstrates how to use the listener mechanism to enforce globally unique field ids (meaning both the model for which code is generated and all its imports recursively).

```java
package mypackage;
 
import java.util.HashSet;
import java.util.Set;
import com.neeve.adm.AdmField;
import com.neeve.adm.AdmModel;
import com.neeve.adm.AdmModelImport;
import com.neeve.adm.AdmSourceCodeErrorAggregator;
import com.neeve.tools.AdmCodeGenerator;
import com.neeve.tools.AdmCodeGenerator.CodeGenerateEvent;
public class CodegenListener implements AdmCodeGenerator.CodegenListener {
    @Override
    public void codeGenerateEvent(CodeGenerateEvent e) {
        switch (e.getEventType()) {
            case START:
                System.err.println("CodegenListener START");
                break;
            case SKIP:
                System.err.println("CodegenListener SKIP");
                break;
            case MODEL_PARSED:
                System.err.println("CodegenListener MODEL_PARSED " + e.getModel().getFullName());
                validateFieldIdsUnique(e.getModel(), e.getErrorAggregator(), new HashSet<Short>(), new HashSet<String>());
                // e.getErrorAggregator().add("Test external error", AdmSourceCodeErrorAggregator.Severity.ERROR, e.getModel().getCodeSource());
                break;
            case END:
                System.err.println("CodegenListener END " + e.getModel().getFullName());
                // e.getErrorAggregator().add("Test external error", AdmSourceCodeErrorAggregator.Severity.ERROR, e.getModel().getCodeSource());
                break;
        }
    }
    private void validateFieldIdsUnique(AdmModel model, AdmSourceCodeErrorAggregator aggregator, Set<Short> fields, Set<String> modelsProcessed) {
        // iterate imports and run validation for them if not already run. We will run bottom -> top validation so we assume that imported models have correct
        // field ids and the importing model doesn't if it defines the same id as in one of the imports.
        for (AdmModelImport modelImport : model.getModelImports()) {
            if (!modelsProcessed.contains(modelImport.getModel().getFullName())) {
                validateFieldIdsUnique(modelImport.getModel(), aggregator, fields, modelsProcessed);
                modelsProcessed.add(modelImport.getModel().getFullName());
            }
        }
        // now we check fields of current model
        for (AdmField field : model.getFields()) {
            if (fields.contains(field.getId())) {
                // aggregator collects errors and they will be displayed at the end of code generation.
                aggregator.add("Duplicate field id",
                               AdmSourceCodeErrorAggregator.Severity.ERROR,
                               field.getCodeSource(), null);
                // note that classes derived from AdmModelElement
                // usually have source code (error line) information retrieved through getCodeSource()
                // pointing back to place in XML where their XML representation is.
            }
            else {
                fields.add(field.getId());
            }
        }
    }
}
```

The listener is provided to the code generator as follows:

```xml
<plugins>
    <!-- Generates X model from XML model file -->
    <plugin>
        <groupId>com.neeve</groupId>
        <artifactId>nvx-core-maven-plugin</artifactId>
        <version>${project.version}</version>
        <executions>
            <execution>
                <id>Model</id>
                <phase>generate-sources</phase>
                <goals>
                    <goal>generate</goal>
                </goals>
                <configuration>
                    <!-- The usual configuration params go here ... -->
 
                    <!-- The listener class that should be on the build classpath for this project -->
                    <codegenListenerClassName>mypackage.CodegenListener</codegenListenerClassName>
                </configuration>
            </execution>
        </executions>
    </plugin>
</plugins>
```

## Incremental Code Regeneration <a href="#generatingsourcecode-incrementalcoderegeneration" id="generatingsourcecode-incrementalcoderegeneration"></a>

The Maven ADM/Platform Plugin and ADM Code generator take a source model's last modified timestamp or checksum into account. Code generation will be skipped if:

* The model file has been changed since the last build.
* Any import model file has changed (checked recursively in imports of imports...)
* Any input option for code generation has changed such as encoding type, namespace, directives etc.

The incremental code generation works by tracking above given changes in an XML file which may be found in output dir. The file has a name with a pattern like this:

`.${model_filename}.xml_${md5checksum}.metadata.` **model\_name** is the name of the model file for which code was generated. **md5checksum** is a signature calculated from input options given to code generator so that if any of them changes, the resulting filename no longer represents same code generation. Stored in this file are input options given to code generator and list of models with a number that would be different every time model file is persisted to disk. These files do not go into the jar and may be deleted at any time (which they usually do when a clean build is triggered).


# Choosing an Encoding Type

## Overview

The ADM code generator supports the following encoding formats for generated classes

* Json
* Protobuf
* Xbuf2

From an API perspective the generated interfaces are functionally equivalent (with a few exceptions), but each encoding has different performance characteristics and serialize to the appropriate encoding format.

## Encoding Types

### Json <a href="#choosinganencoding-json" id="choosinganencoding-json"></a>

The `Json` encoding generates fairly simple classes that serialize to/from json. This encoding type is suitable for lightweight applications or for applications that natively work with json (e.g. web applications).

#### **Pros**

* Memory utilization
  * Because there isn't much serialization machinery or caching of the backing serialized format, Json generated objects don't use much memory which can be useful for long lived state objects.

#### **Cons**

* Performance
  * JSON serialization is slow and produces a lot of garbage, and JSON objects can't be pooled.
* Size
  * Serializing to text is not very compact which leads to higher disk usage and network bandwidth.

{% hint style="warning" %}
The `Json` encoding type is going to be removed from the next X Platform major release. Please see [Encoding Types](/talon/developing-applications/modeling-messages-and-state/the-code-generator) for more information
{% endhint %}

### Protobuf <a href="#choosinganencoding-protobuf" id="choosinganencoding-protobuf"></a>

With protobuf encoding objects are create with backing google protobuf generated objects. Protobuf is suitable for applications with higher performance requirements than is afforded by Json encoding. It should be used by applications with moderate to high performance requirements.

Protobuf is recommended for generating ADM objects used for the microservice store.

#### **Pros**

* Memory Utilization
  * Protobuf generated objects are fairly compact in memory compared to Xbuf objects, non repeated field values are store directly in the generated message object, making protobuf encoded objects a good candidate for usage as state entities.
* Performance
  * Faster serialization than Json.
* Interoperability
  * Protobuf is a well known standard, making it easy to interoperate with applications not uses ADM generated code.

#### **Cons**

* Performance Predictability
  * Google protobuf generated messages are not zero garbage and, thus, can result in large garbage collection related pauses.

{% hint style="warning" %}
The `Protobuf` encoding type is going to be removed from the next X Platform major release in favor of the `Xbuf2` encoding type. Please see [Encoding Types](/talon/developing-applications/modeling-messages-and-state/the-code-generator) for more information
{% endhint %}

### Xbuf2 <a href="#choosinganencoding-xbuf" id="choosinganencoding-xbuf"></a>

Xbufs generated objects, backed by Talon's high performance implementation of Google protobufs, supports zero garbage operation and cut-through serialization (the ability to read/write fields directly to from a backing buffer). It should be used for applications with the most stringent performance requirements.

Xbuf2 is recommended for use with ADM message models particularly for applications that require very low latency.

#### **Pros**

* Performance (Throughput & Latency)
  * Faster serialization than `Json` or `Protobuf` encoding types
  * Optimized for both messages and state
* Lower Memory Footprint
  * Object recycling and zero garbage support results in lower memory footprint than the `Json` and `Protobuf` encoding types
* Interoperability
  * Protobuf is a well known standard, making it easy to interoperate with applications not uses ADM generated code.
* Tunability
  * Offers several knobs to manage the tradeoff between performance and memory conservation
* Flexibility
  * Offers multiple data access patterns
    * Random access (as is offered by the other encoding types)
    * Serial access
      * Direct Deserialization: The ability to serially traverse a Google Protobuf encoded buffer and dispatch the fields to the application via a callback
      * Direct Serialization: The ability for an application to directly serialize application fields into a buffer in the Google Protobuf wire format

#### **Cons**

* Complexity
  * Xbuf2 stores field data off-heap. This can result in more complexities in the following areas
    * Troubleshooting issues
    * Monitoring memory utilization
    * Performing capacity planning particularly related to memory utilization.
  * Xbuf2 generated classes are larger
  * Working with pooling can result in higher development complexity

#### **Known Limitations**

* Does not support the following field types
  * UUID
  * UUID\[]
  * Currency
  * Currency\[]

## API Differences

For the most part code generated for the different encoding types behaves the same, but there are some key differences that stem from both the underlying serialization mechanisms and features supported.

### Unrecognized Field Values <a href="#choosinganencoding-unrecognizedfieldvalues" id="choosinganencoding-unrecognizedfieldvalues"></a>

1. For Json encoding unrecognized enum array values are treated as null, and for non array fields an unrecognized array value will be treated as null and hasXXX will return true.
2. For Protobuf and Xbuf2, unrecognized fields (those with unrecognized field tags) are preserved when an inbound message is written to a transaction log (although they are inaccessible). If the message is copied by serializing to bytes and deserializing into a new message instance, the unrecognized fields from the original message are sent on the wire. If the message is modified prior to sending, the unrecognized fields may be lost.
3. For repeated enum fields in Protobuf, unrecognized enum values are ignored. For Protobuf encoding the underlying protobuf may reorder the unrecognized enum values and put them at the end. Xbuf2 generated code preserves the order of unrecognized enums. When deserializing from Json, unrecognized enum values are treated as null so the effect on a deserialized message or entity is the same as adding an enum array with null values (see below).

### Null Value Handling <a href="#choosinganencoding-nullvaluehandling" id="choosinganencoding-nullvaluehandling"></a>

1. Classes generated with Json encoding support serializing null values and null values in arrays.
2. For Protobuf and Xbuf2, setting a null value for a String, Date, Enum or Embedded Entity Field results in the field being cleared (the Google Protobuf wire format doesn't support null values on the wire).
3. For Protobuf, setting a Date\[], String\[], or Enum\[] containing a null element results in a NullPointerException being thrown. For Xbuf2, the behavior is the same as with Entity\[] i.e. the null values are ignored.
4. For Protobuf and Xbuf2, setting an Entity\[] with a null element results in the null value(s) being ignored during serialization. The same holds true when using the XIterator setters or when calling addXXX to add the set of values.
5. For Protobuf, after setting null values in an array field, subsequently calling the getter *MAY* or *MAY NOT* result in the null values being returned. Applications are encouraged to use the getXXXIterator accessors, and should be coded to handle either case for maximum portability both between encodings and for handling cases where the null values have been filtered out due to serialization. For Xbuf2, a subsequent call to get or iterator over array elements after setting a null element will NOT return the null element.

### Pooling Considerations <a href="#choosinganencoding-poolingconsiderations" id="choosinganencoding-poolingconsiderations"></a>

A major difference between Xbuf2 and Protobuf or Json encoded entities is that Xbuf2 messages and entities are pooled by the platform by default. From a coding standpoint this means that when working with Xbuf2 encoded messages or entities:

1. An application may not hold onto an Xbuf2 encoded message beyond the scope of a message handler.
2. An application may not hold onto an XString or embedded entity type from a message beyond the scope of a message handler because these objects are pooled along with the message and will be reset once the message is returned to its pool. See Zero Garbage Nested Entities for detailed usage, but the general rule of thumb is to copy any entity that needs to be retained in the microservice store, or to use the more advanced 'take' apis. Note that 'take' is not supported for String fields as string fields are not pooled for Xbuf2 messages.
3. Setting an XString or embedded entity field on a message transfers ownership to the message. If the application wants to retain the entity in the microservice store, then it should copy it into a new entity or use the more advanced 'lend' apis. See Zero Garbage Nested Entities for more details.
4. An application may not mutate a returned array type from a message and should not hold onto to the array beyond the duration of a message handler. See Zero Garbage Array Accessors for more details.

\\

\\


# Configuring Messaging

Talon microservices are message driven services. To receive messages, a microservice connects to one or more messaging buses and register message interest on the buses so that messages published on the bus are drawn to the microservice and dispatched to message handlers for processing. This section describes how to define the abstract bus structure globally and how to configure microservice-specific connections and message interest.

This section requires a knowledge of the key messaging related abstractions as discussed in the [Talon Messaging Model](/talon/concepts-and-architecture/messaging-model). The following is a quick recap of these abstractions.

* A **message bus** groups messaging participants and serves as a container of one or more **message channels**. Message channels support one-many communication semantics and serve as named conduits through which messaging participants exchange messages.
* Message channels define the concept of **channel keys** and **channel filters**. Keys and filters are used to facilitate fine-grained message routing on the bus. Channel keys are configured bus wide while channel filters are configured on a per participant basis. For each message sent on a channel, the channel's key is used in conjunction with message contents and **key resolution tables** **(KRT)** to resolve the **message key** for the sent messages. A sent message is delivered to participants with filters that match the message's key.
* To participate in message exchange, a Talon microservice connects to one or more messaging buses, configures and **joins** each of the channels it is interested in receiving messages from. Once messaging is started, the microservice processes Inbound messages in message handlers and sends outbound messages through channels.

The AEP engine uses configuration information to manage the lifecycle and policies of messaging connections and register messaging interest (configure and join channels) on behalf of its microservice. Once messaging interest has been registered and the messaging connections created and started, the engine receives and dispatches messages inbound on joined channels to the microservice's message handlers and provides mechanisms to the microservice to send outbound messages.

## Configuration Organization

Messaging configuration is split between **global** (shared) and **per-microservice** concerns:

**Global Configuration** (in `<buses>` section):

* Bus names and structure
* Channel catalog (channel names, QoS, key patterns)

**Per-Microservice Configuration**:

* **Bus connection descriptors** - Though physically specified in the `<buses>` section (typically using substitution parameters), connection descriptors are conceptually per-microservice as each microservice may require different connection credentials, client IDs, or other provider-specific parameters
* **Message factories** - Which ADM-generated factories the microservice uses (in `<app>` section)
* **Channel joins and filters** - Which channels to join and with what subscription filters (in `<app>` section)
* **Subscription policies** - Preservation behavior on shutdown (in `<app>` section)

### Topics

* [**Configuring Bus Connections**](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - Define the abstract bus structure globally and configure per-microservice connection descriptors
* [**Registering Message Interest**](/talon/developing-applications/configuring-messaging/registering-message-interest) - Configure microservices to join channels, register message factories, and define message handlers

{% hint style="info" %}
This section focuses exclusively on configuring message buses and bus connections required by the AEP engine and Talon runtime. These configurations support functions such as connection establishment, lifecycle management, inbound message receipt and dispatch, and outbound message sends. It does not cover programmatic message handling or sending, which are implemented by the microservice developer. Those topics are discussed later in [Authoring User Code](/talon/developing-applications/authoring-user-code).
{% endhint %}


# Configuring Bus Connections

Message bus configuration is split between **global** (shared structure) and **per-microservice** (connection details) concerns.

## Understanding Configuration Scope

**Global Configuration** defines the abstract bus structure shared across microservices:

* Bus names
* Channel catalog (channel names, QoS, key patterns)

**Per-Microservice Configuration** defines connection details specific to each microservice:

* **Connection descriptors** - Provider, address, credentials, and connection properties

{% hint style="info" %}
**Connection Descriptor Placement**: While connection descriptors are conceptually per-microservice configuration, they are physically specified in the global `<buses>` section. This allows most descriptor properties to be shared while using **substitution parameters** to provide microservice-specific values (such as credentials or client IDs) in each microservice's configuration.
{% endhint %}

## Global Bus Structure

The abstract bus structure is defined in the `<buses>...</buses>` section of the configuration DDL. This structure defines the bus name and the catalog of available channels.

### Bus Properties

The following properties define the global bus structure:

<table><thead><tr><th width="137.98712158203125">Property</th><th width="190.28448486328125">Required/Optional</th><th>Description</th></tr></thead><tbody><tr><td>name</td><td>Required</td><td>The (unique) name of the bus. This name is used by microservices to reference the bus.</td></tr><tr><td>displayName</td><td>Optional, default=null</td><td>The (non-unique) display name of the bus.</td></tr><tr><td><strong>enabled</strong></td><td>Optional, default=true</td><td>Used to enable/disable the bus.</td></tr><tr><td><strong>channels</strong></td><td>Optional</td><td>The set of channels contained in the bus (the channel catalog). See below for channel properties.</td></tr></tbody></table>

### Channel Properties

The following are the set of global bus channel properties.

<table><thead><tr><th width="136.3797607421875">Property</th><th width="267.55499267578125">Required/Optional</th><th>Description</th></tr></thead><tbody><tr><td><strong>id</strong></td><td>Optional, default=&#x3C;not present></td><td>The channel id. If present, it must be unique across all channels in the bus since it is used, in lieu of the channel name to uniquelt identify the channel in the bus system wide i.e., it is used, in lieu, of the name in the SMA metadata on the wire to identify the channel on which a message has travelled on the bus.</td></tr><tr><td><strong>qos</strong></td><td>Optional, default=Guaranteed</td><td>The channel quality of service. The value of this property must be either <strong>BestEffort</strong> or <strong>Guaranteed</strong></td></tr><tr><td><strong>key</strong></td><td>Optional, default=null</td><td>The channel key</td></tr></tbody></table>

## Connection Descriptor

The connection descriptor specifies how a microservice connects to the underlying messaging bus. **Connection descriptors are conceptually per-microservice configuration** - each microservice instance typically requires its own connection values (credentials, client IDs, provider-specific settings).

While physically specified in the global `<buses>` section, connection descriptors use **substitution parameters** to enable per-microservice values. The actual values are provided in the `<xvm>` section's `<env>` elements, allowing different microservice instances to connect with different credentials while sharing the same bus structure.

### Descriptor Format

A connection descriptor can be specified in two forms:

{% tabs %}
{% tab title="Non-Decomposed Format" %}
The descriptor can be specified as a single connection string using the format:

```
<provider>://<address>[:<port>][&prop1=value1][&prop2=value2]...
```

**Example:**

```xml
<bus name="sample-bus" descriptor="activemq://localhost:61616&set_key_on_receipt=true">
  <channels>
    <channel id="1" name="new-orders-channel">
      <qos>Guaranteed</qos>
      <key>NEWORDERS/${Region}/${Department}</key>
    </channel>
  </channels>
</bus>
```

{% endtab %}

{% tab title="Decomposed Format" %}
The descriptor can be decomposed into separate DDL elements:

```xml
<bus name="sample-bus">
  <provider>activemq</provider>
  <address>localhost</address>
  <port>61616</port>
  <properties>
    <set_key_on_receipt>true</set_key_on_receipt>
  </properties>
  <channels>
    <channel id="1" name="new-orders-channel">
      <qos>Guaranteed</qos>
      <key>NEWORDERS/${Region}/${Department}</key>
    </channel>
  </channels>
</bus>
```

{% endtab %}
{% endtabs %}

Both formats support substitution parameters for per-microservice values:

```xml
<bus name="orders-bus">
  <provider>solace</provider>
  <address>${solace.host}</address>
  <port>${solace.port}</port>
  <properties>
    <username>${solace.username}</username>
    <password>${solace.password}</password>
  </properties>
</bus>
```

### Provider-Agnostic Properties

The following connection properties are supported across all messaging providers.

#### Message Receipt

| Property                           | Default | Description                                                                                                                                                                                                                                                                                                                                                                                         |
| ---------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `set_bus_and_channel_on_receipt`   | false   | Controls whether the bus and channel name are set on received messages. Setting the channel and bus name on inbound messages incurs performance overhead. Performance sensitive applications should avoid enabling this property.                                                                                                                                                                   |
| `set_key_on_receipt`               | false   | Controls whether the message key is set on received messages. Not all binding implementations transport the key on the wire; this property has no effect for bindings that don't transport the key. Setting the key on inbound messages incurs a performance overhead. Performance sensitive applications should avoid enabling this property.                                                      |
| `set_sno_on_receipt`               | true    | Controls whether the message sequence number is set on received messages. If set to false, inbound messages surfaced to the AEP engine and application will not have the sequence number set regardless of whether the sender was configured to set sequence numbers in outbound messages. Incurs a slight performance overhead when enabled.                                                       |
| `enable_inbound_transport_headers` | false   | Controls whether transport headers are set in inbound messages. Setting this to true causes bindings that support this functionality to set transport-specific headers in the metadata section of inbound messages. Not all binding implementations support this. Enabling this fosters tighter coupling between the application and specific message bindings, which is generally not recommended. |

#### Topic Resolution

| Property                    | Default | Description                                                                                                                                                                                  |
| --------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `topic_starts_with_channel` | true    | Controls whether topic names start with the channel name for the bus. For bus bindings that support topic routing, this controls whether the resolved key is prefixed with the channel name. |

#### Channel Resolution

| Property                    | Default | Description                                                                                                                                                                                                        |
| --------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `auto_add_catchall_channel` | false   | Controls whether a catchall channel is automatically added to the bus descriptor when the binding is created. When enabled, a catchall channel is added provided one is not already present in the bus descriptor. |

#### Message Sending

| Property                  | Default | Description                                                                                                  |
| ------------------------- | ------- | ------------------------------------------------------------------------------------------------------------ |
| `enable_concurrent_sends` | false   | Controls whether sends through the message bus binding can be performed concurrently using multiple threads. |

#### Configuration

| Property                     | Default | Description                                                                                                                                                                                                                                                                                                                                                               |
| ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `additional_properties_file` | null    | Specifies the path to an external file used to load additional bus configuration properties. The external file format is a plain Java Properties file. Properties specified in the external file will be merged into the configuration property set for the bus. This file is loaded at runtime from the local file system on the host where the configured XVM will run. |

### Provider-Specific Properties

In addition to the provider-agnostic properties listed above, each messaging binding supports additional configuration properties specific to its underlying messaging technology.

For complete configuration reference and architectural information about each binding, see:

* [**Solace Binding**](/talon/developing-applications/configuring-messaging/configuring-bus-connections/solace-binding) - Configuration reference | [Concepts](/talon/concepts-and-architecture/messaging-model/solace-binding)
* [**JMS Binding**](/talon/developing-applications/configuring-messaging/configuring-bus-connections/jms-binding) - Configuration reference | [Concepts](/talon/concepts-and-architecture/messaging-model/jms-binding)
* [**Loopback Binding**](/talon/developing-applications/configuring-messaging/configuring-bus-connections/loopback-binding) - Configuration reference | [Concepts](/talon/concepts-and-architecture/messaging-model/loopback-binding)
* [**Executor Binding**](/talon/developing-applications/configuring-messaging/configuring-bus-connections/executor-binding) - Configuration reference | [Concepts](/talon/concepts-and-architecture/messaging-model/executor-binding)

## Providing Per-Microservice Connection Values

Since connection descriptors are conceptually per-microservice configuration, each microservice typically needs to provide its own connection values (such as credentials, client IDs, or other provider-specific parameters). This is accomplished using **substitution parameters** in the global bus descriptor, with values provided in the `<xvm>` section where the microservice runs.

**Example using substitution parameters**:

```xml
<model>
  <!-- Global bus structure with parameterized connection descriptor -->
  <buses>
    <bus name="orders-bus">
      <provider>solace</provider>
      <address>${solace.host}</address>
      <port>${solace.port}</port>
      <properties>
        <vpn>${solace.vpn}</vpn>
        <username>${solace.username}</username>
        <password>${solace.password}</password>
      </properties>
      <channels>
        <channel name="new-orders">
          <qos>Guaranteed</qos>
          <key>ORDERS/${Region}</key>
        </channel>
      </channels>
    </bus>
  </buses>

  <!-- Application definitions (business logic) -->
  <apps>
    <app name="OrderProcessor" mainClass="com.example.OrderProcessor">
      <messaging>
        <bus name="orders-bus">
          <channels>
            <channel name="new-orders" join="true"/>
          </channels>
        </bus>
      </messaging>
    </app>

    <app name="InventoryService" mainClass="com.example.InventoryService">
      <messaging>
        <bus name="orders-bus">
          <channels>
            <channel name="new-orders" join="true"/>
          </channels>
        </bus>
      </messaging>
    </app>
  </apps>

  <!-- XVMs (containers) provide microservice-specific connection values -->
  <xvms>
    <xvm name="order-processor-1">
      <env>
        <solace.host>solace-prod.example.com</solace.host>
        <solace.port>55555</solace.port>
        <solace.vpn>prod-vpn</solace.vpn>
        <solace.username>order-processor-user</solace.username>
        <solace.password>${ORDER_PROCESSOR_PASSWORD}</solace.password>
      </env>
      <apps>
        <app name="OrderProcessor" autoStart="true"/>
      </apps>
    </xvm>

    <xvm name="inventory-service-1">
      <env>
        <solace.host>solace-prod.example.com</solace.host>
        <solace.port>55555</solace.port>
        <solace.vpn>prod-vpn</solace.vpn>
        <solace.username>inventory-service-user</solace.username>
        <solace.password>${INVENTORY_SERVICE_PASSWORD}</solace.password>
      </env>
      <apps>
        <app name="InventoryService" autoStart="true"/>
      </apps>
    </xvm>
  </xvms>
</model>
```

In this example:

* The global `<buses>` section defines the bus descriptor with substitution parameters like `${solace.username}` and `${solace.password}`
* The `<apps>` section defines the microservice business logic and which channels they join
* Each `<xvm>` (Talon container) provides its own connection values via `<env>` elements
* When an XVM starts, it resolves the substitution parameters using its `<env>` values, allowing each microservice instance to connect with different credentials
* This allows the abstract bus structure (channels, QoS, keys) to be shared while connection credentials remain microservice-specific


# Solace Binding

Configuration reference for the Solace message bus binding.

## Overview

The Solace binding provides native integration with Solace PubSub+ message brokers using either Solace's JCSMP (Java) API or CCSMP (C via JNI) API. The JNI binding supports zero-garbage messaging in steady state but is Linux-only, while the Java binding works on all platforms.

For conceptual information about the Solace binding, see [Solace Binding](/talon/concepts-and-architecture/messaging-model/solace-binding).

## JNI vs Java Binding Selection

Control which Solace client library is used:

| Property    | Default | Description                                                                               |
| ----------- | ------- | ----------------------------------------------------------------------------------------- |
| `usejni`    | -       | When `true`, mandates JNI usage (fails if unavailable). When `false`, never uses JNI.     |
| `preferjni` | true    | When `true`, silently falls back to JCSMP if JNI unavailable. Ignored if `usejni` is set. |

{% hint style="info" %}
The JNI binding is only available on Linux platforms and provides zero-garbage messaging capabilities.
{% endhint %}

## Bus Descriptor Format

Solace buses can be configured using a descriptor string or decomposed DDL format.

### Descriptor String

```
solace://<address>:<port>&prop1=val1&propN=valN
```

**Example**:

```
solace://192.168.1.9:55555&usejni=true&vpn_name=default&username=test&password=test
```

### Decomposed DDL Format

```xml
<buses>
  <bus name="my-bus">
    <provider>solace</provider>
    <address>192.168.1.9</address>
    <port>55555</port>
    <properties>
      <usejni>true</usejni>
      <vpn_name>default</vpn_name>
      <username>test</username>
      <password>test</password>
    </properties>
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

### As Descriptor (Substitution Support)

```xml
<buses>
  <bus name="my-bus" descriptor="solace://${solace.host}:${solace.port}&vpn_name=${solace.vpn}&username=${solace.user}&password=${solace.password}">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

## Solace Provider Properties

### Passed Through Properties

Provider properties specified in the bus descriptor are passed through to the Solace connection with the following processing:

1. Properties described in [`SolaceBindingProperties`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/solace/SolaceBindingProperties.html) are stripped out
2. General bus properties from [`MessageBusDescriptor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageBusDescriptor.html) are stripped out
3. Rationalized properties (see below) are translated based on JNI vs Java binding and passed through
4. If `usejni=true`: Properties not starting with `FLOW_` or `SESSION_` are stripped, rest passed through
5. If `usejni=false`: Properties not starting with `jcsmp` are stripped, rest passed through

{% hint style="warning" %}
Neeve does not test properties not explicitly documented on this page or in the javadoc. End users should test any passed-through properties thoroughly.
{% endhint %}

### Rationalized Passed Through Properties

These common properties are automatically translated between JCSMP (Java) and CCSMP (JNI):

| Binding Property             | Default    | JCSMP Property                                     | CCSMP Property                          |
| ---------------------------- | ---------- | -------------------------------------------------- | --------------------------------------- |
| `vpn_name`                   | -          | `jcsmp.vpn_name`                                   | `SESSION_VPN_NAME`                      |
| `username`                   | vpn\_name  | `jcsmp.username`                                   | `SESSION_USERNAME`                      |
| `password`                   | username   | `jcsmp.password`                                   | `SESSION_PASSWORD`                      |
| `publish_window_size`        | 255        | `jcsmp.pub_ack_window_size`                        | `SESSION_PUB_WINDOW_SIZE`               |
| `reconnect_retry_count`      | 100        | `jcsmp.CLIENT_CHANNEL_PROPERTIES.ReconnectRetries` | `SESSION_RECONNECT_RETRIES`             |
| `connect_retry_count`        | 3          | `jcsmp.CLIENT_CHANNEL_PROPERTIES.ConnectRetries`   | `SESSION_CONNECT_RETRIES`               |
| `tcp_nodelay`                | auto-tuned | `jcsmp.CLIENT_CHANNEL_PROPERTIES.tcpNoDelay`       | `SESSION_TCP_NODELAY`                   |
| `reapply_subscriptions`      | true       | `jcsmp.REAPPLY_SUBSCRIPTIONS`                      | `SESSION_REAPPLY_SUBSCRIPTIONS`         |
| `ignore_subscriptions_error` | true       | `jcsmp.IGNORE_DUPLICATE_SUBSCRIPTION_ERROR`        | `SESSION_IGNORE_DUP_SUBSCRIPTION_ERROR` |
| (address from descriptor)    | -          | `jcsmp.HOST`                                       | `SESSION_HOST`                          |

### Additional Properties

See [`SolaceBindingProperties`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/solace/SolaceBindingProperties.html) javadoc for the complete list of binding-specific properties.

### Auto Tuning

When not explicitly set, these properties are automatically configured based on `nv.optimizefor`:

| Binding Property                        | `nv.optimizefor=latency`                                                                                                                                              | `nv.optimizefor=throughput`                                                                                                                                           | No Optimization |
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- |
| `tcp_nodelay`                           | true                                                                                                                                                                  | false                                                                                                                                                                 | false           |
| `detached_sends_queue_wait_strategy`    | See [`XRuntime.createWaitStrategy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html#createWaitStrategy\(java.lang.String,%20boolean\)) | See [`XRuntime.createWaitStrategy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html#createWaitStrategy\(java.lang.String,%20boolean\)) | Blocking        |
| `detached_dispatch_queue_wait_strategy` | See [`XRuntime.createWaitStrategy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html#createWaitStrategy\(java.lang.String,%20boolean\)) | See [`XRuntime.createWaitStrategy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ci/XRuntime.html#createWaitStrategy\(java.lang.String,%20boolean\)) | Blocking        |

## Session Configuration

By default, the binding creates two separate sessions — one for consuming (suffixed with `-IN`) and one for producing (suffixed with `-OUT`) — to avoid network flow control deadlocks and improve throughput.

| Property                      | Default                | Description                                                                                                                                                                                                                                         |
| ----------------------------- | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `single_session`              | false                  | Use a single session for both publishing and consuming. Default uses separate consumer and producer sessions to avoid network flow control deadlocks. Single session uses fewer threads and resources and is suitable for unidirectional messaging. |
| `client_name`                 | username or queue name | Unique client name to identify the session on the appliance. Suffixed with `-IN` and `-OUT` unless `single_session=true`. If `use_default_queue_name_as_default_client_id` is set, the default queue name is used instead.                          |
| `session_open_retry_count`    | 5                      | Number of retry attempts to establish a session after transient errors. Permanent errors (e.g., invalid client name, dynamic clients not allowed, endpoint property mismatch) are not retried.                                                      |
| `session_open_retry_interval` | 1                      | Interval in seconds between session open retry attempts.                                                                                                                                                                                            |
| `consumer_cpu_affinity_mask`  | 0                      | CPU affinity mask for the Solace consumer session's Context receiver thread. Not to be confused with the detached dispatcher thread.                                                                                                                |
| `producer_cpu_affinity_mask`  | 0                      | CPU affinity mask for the Solace producer session's Context receiver thread. Not to be confused with the detached sender thread.                                                                                                                    |

## Queue Configuration

A Solace queue must be provisioned for the binding to support guaranteed messaging. The queue holds messages for the application while it is disconnected.

| Property                                      | Default                 | Description                                                                                                                                                                                         |
| --------------------------------------------- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `queue_name`                                  | X-SMA-{name}-{userName} | Solace queue name for guaranteed messaging. When not specified, a default name is generated from the bus name and user name. When set to blank (empty string), no queue is used.                    |
| `use_default_queue_name`                      | true                    | Whether to use the auto-generated default queue name when `queue_name` is not specified. Disabling this without setting `queue_name` means no queue is used, compromising guaranteed messaging QoS. |
| `use_default_queue_name_as_default_client_id` | false                   | Use the default queue name as the client name when `client_name` is not explicitly set.                                                                                                             |
| `provision_queue`                             | true                    | Attempt to provision the queue on a best-effort basis. Requires endpoint management to be enabled on the appliance. If the queue cannot be provisioned, a warning is logged.                        |
| `queue_quota`                                 | 2048                    | Queue quota in MB when provisioning a queue (default 2 GB).                                                                                                                                         |

## Detached Sends

The binding can create a detached sender thread to offload message marshalling and Solace API send calls from the application thread, improving throughput.

{% hint style="info" %}
Talon AepEngine users should note that this is separate from the detached send specified for the AEP bus. Enabling this property in conjunction with detached send in AEP results in two detached send threads.
{% endhint %}

| Property                             | Default    | Description                                                                                                                                           |
| ------------------------------------ | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `detached_sends`                     | false      | Create a detached sender thread for published messages. Offloads message marshalling and Solace API calls to improve throughput.                      |
| `detached_sends_queue_depth`         | 1024       | Disruptor ring buffer queue depth for the detached sender.                                                                                            |
| `detached_sends_queue_wait_strategy` | auto-tuned | Wait strategy for the detached sends disruptor queue (e.g., `BusySpinWaitStrategy`, `YieldingWaitStrategy`, `BlockingWaitStrategy`). See Auto Tuning. |
| `detached_sends_cpu_affinity_mask`   | 0          | CPU affinity mask for the detached sender thread.                                                                                                     |

## Detached Dispatch

The binding can create a detached dispatch thread for message deserialization and dispatch, allowing the Solace receiver thread to exclusively handle I/O work.

| Property                                | Default    | Description                                                                                                                                              |
| --------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `detached_dispatch`                     | false      | Create a detached dispatch thread for received messages. Offloads deserialization and dispatch from the Solace receiver thread to improve throughput.    |
| `detached_dispatch_queue_depth`         | 128        | Disruptor ring buffer queue depth for the detached dispatcher.                                                                                           |
| `detached_dispatch_queue_wait_strategy` | auto-tuned | Wait strategy for the detached dispatch disruptor queue (e.g., `BusySpinWaitStrategy`, `YieldingWaitStrategy`, `BlockingWaitStrategy`). See Auto Tuning. |
| `dispatcher_cpu_affinity_mask`          | 0          | CPU affinity mask for the detached dispatcher thread.                                                                                                    |

## Wire Metadata

| Property               | Default | Description                                                                                                                                                                                                                                                                                                                  |
| ---------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sma_metadata_version` | 1       | Controls the version of SMA metadata encoded in outbound messages. The Solace binding encodes SMA message metadata in the message's SDTMap keyed by `x-sma-metadata`. Use `2` for more efficient metadata if no receivers use versions prior to 1.8.396. The metadata version must be supported by all downstream receivers. |

## Message Handling

| Property                                 | Default | Description                                                                                                                                                                |
| ---------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `set_key_on_receipt`                     | false   | Sets the topic on a per-message basis. Accessible via `message.getMessageKeyAsRaw()`.                                                                                      |
| `fail_on_inbound_message_handling_fault` | false   | Fail the binding unconditionally when an inbound message handling fault occurs. When `false`, an `UnhandledMessageEvent` is dispatched, leaving the application to decide. |
| `treat_non_x_inbound_as_fault`           | true    | Treat receipt of non-X (non-SMA) messages as a fault. When `false`, non-X messages are handled gracefully.                                                                 |

## Orphan Subscription Checks

When a Solace bus binding has a queue name specified, it can detect "orphan" subscriptions - subscriptions on the queue that don't match those issued by the application (presumably left over from an earlier session).

### Policies

| Policy                  | Description                                                                                                                                      |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| None                    | Appliance is not queried for subscriptions (default).                                                                                            |
| Ignore                  | Query appliance and trace orphan subscriptions at info level. Report to subscription validator if registered.                                    |
| LogExceptionAndContinue | Query appliance and log exception if orphans found.                                                                                              |
| NoOrphan                | Query appliance and throw exception from binding start method if orphans found.                                                                  |
| Unsubscribe             | With this policy enabled, the appliance is queried for subscriptions and orphan subscriptions are then removed (unsubscribed) from the appliance |

### Configuration

To perform subscription checks, SEMP must be enabled:

| Property                               | Usage       | Default       | Description                                                                                                                                                                                                                                                      |
| -------------------------------------- | ----------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enable_semp`                          | optional    | false         | Enable Solace SEMP over messaging requests. Required for subscription checks. Requires appliance has SEMP over messaging enabled and user is authorized. **NOTE**: SEMP operations are experimental and not recommended for production without rigorous testing. |
| `orphan_subscription_check`            | optional    | None          | Orphan subscription check policy. Performed after subscriptions created but before messaging started. Requires `enable_semp=true` for policies other than None.                                                                                                  |
| `orphan_subscription_check_batch_size` | optional    | 200           | Batch size for sequenced fetch of subscriptions during orphan subscription check. Used when the subscription count exceeds what can be returned in a single response.                                                                                            |
| `discard_semp_prestart_messages`       | optional    | true          | Discard BestEffort messages received before SEMP checks complete during binding startup. Prevents the dispatch thread from blocking on SEMP requests. Only disable if the bus's event handler will not block on messages received before `start()` returns.      |
| `semp_version`                         | recommended | soltr/7\_1\_1 | SEMP version. Must match messaging appliance version. If not set, version auto-detected on first request.                                                                                                                                                        |
| `semp_request_timeout`                 | recommended | 10000         | Timeout for SEMP requests in milliseconds. Test under load to ensure sufficient.                                                                                                                                                                                 |
| `subscription_validator`               | unsupported | -             | Legacy property for custom subscription validation. Use only with Neeve support guidance.                                                                                                                                                                        |

{% hint style="warning" %}
**SEMP Operations Warning**: SEMP operations are experimental. Not recommended for production without rigorous end-user testing. Solace may drop SEMP over messaging support in future versions. Reliance on SEMP may limit ability to connect to appliances with incompatible SEMP models.
{% endhint %}

## Enforcing Max Queue Bind Count

When using Guaranteed messaging, queues can be provisioned with max bind count of 1 to ensure only one application instance can bind. The binding can enforce this via SEMP:

| Property                 | Usage    | Default | Description                                                                                                                                                                                                                                                                                                                |
| ------------------------ | -------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enable_semp`            | optional | false   | Must be enabled (see above).                                                                                                                                                                                                                                                                                               |
| `enforce_max_bind_count` | optional | 0       | Set to positive value to enforce match against provisioned queue's max bind count. When set, binding issues SEMP request at start to verify queue's max bind count matches this value. Ensures exclusive connectivity with `enforce_max_bind_count=1`. Ignored if no queue configured or value ≤ 0. Requires SEMP enabled. |

{% hint style="info" %}
**Since 3.8**: `enforce_max_bind_count` provides protection against network partitioning by ensuring two instances cannot both consume messages.
{% endhint %}

## Trace Logging

The Solace binding logs trace messages using two loggers:

### The `nv.sma` Logger

Controls trace level and messages emitted by the binding code itself (not the Solace client library).

### The `nv.sol` Logger

Controls trace level and messages emitted by the Solace client runtime.

#### JNI Enabled (CCSMP)

When using JNI, the binding:

1. Maps `nv.sol` logger level to CCSMP trace level (see table below)
2. Registers callback with CCSMP to intercept trace messages
3. Logs CCSMP messages via `nv.sol` logger

**nv.sol to CCSMP Mapping**:

| nv.sol Level                   | CCSMP Level |
| ------------------------------ | ----------- |
| SEVERE                         | ERROR       |
| WARNING                        | WARNING     |
| INFO, CONFIG                   | NOTICE      |
| FINE, FINER, DIAGNOSE, VERBOSE | INFO        |
| FINEST, DEBUG, ALL             | DEBUG       |

**CCSMP to nv.sol Mapping** (for logged messages):

| CCSMP Level     | nv.sol Level |
| --------------- | ------------ |
| CRITICAL, ERROR | SEVERE       |
| WARNING         | WARNING      |
| NOTICE          | INFO         |
| INFO            | FINE         |
| DEBUG           | FINEST       |

#### JNI Disabled (JCSMP)

When using JCSMP, the binding:

1. Maps `nv.sol` logger level to set `com.solacesystems.jcsmp` logger level
2. Assumes all JCSMP logging uses the `com.solacesystems.jcsmp` logger

**nv.sol to com.solacesystems.jcsmp Mapping**:

| nv.sol Level              | com.solacesystems.jcsmp Level |
| ------------------------- | ----------------------------- |
| SEVERE                    | SEVERE                        |
| WARNING                   | WARNING                       |
| INFO                      | INFO                          |
| CONFIG                    | CONFIG                        |
| FINE                      | FINE                          |
| FINER                     | FINER                         |
| DIAGNOSE, VERBOSE, FINEST | FINEST                        |
| DEBUG, ALL                | ALL                           |

See [Trace Logging](/talon/developing-applications/authoring-user-code/trace-logging) for information on configuring loggers.

## See Also

* [Solace Binding](/talon/concepts-and-architecture/messaging-model/solace-binding) - Conceptual overview
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration
* [Solace Documentation](https://solace.com/products/message-routers) - Solace PubSub+ information


# JMS Binding

Configuration reference for the JMS message bus binding.

## Overview

The JMS binding works with JMS 1.1 level JMS clients and is configured using JNDI lookup. The platform also provides provider-specific implementations optimized for ActiveMQ and Tibco EMS.

For conceptual information about the JMS binding, see [JMS Binding](/talon/concepts-and-architecture/messaging-model/jms-binding).

## Bus Descriptor Format

JMS buses can be configured using a descriptor string or decomposed DDL format.

### Descriptor String

```
jms://<address>:<port>&prop1=val1&propN=valN
```

**Example** (Tibco EMS):

```
jms://tibcohost:2732&username=admin&password=changeme&jndi=true&jndi_contextfactory=com.tibco.tibjms.naming.TibjmsInitialContextFactory&jndi_principal=admin&jndi_credentials=changme&jndi_connectionfactory=CSTopicConnectionFactory
```

### Decomposed DDL Format

```xml
<buses>
  <bus name="my-bus">
    <provider>jms</provider>
    <address>192.168.1.9</address>
    <port>55555</port>
    <properties>
      <username>admin</username>
      <password>changeme</password>
      <jndi>true</jndi>
      <jndi_contextfactory>com.tibco.tibjms.naming.TibjmsInitialContextFactory</jndi_contextfactory>
      <jndi_principal>admin</jndi_principal>
      <jndi_credentials>changme</jndi_credentials>
      <jndi_connectionfactory>CSTopicConnectionFactory</jndi_connectionfactory>
    </properties>
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

### As Descriptor (Substitution Support)

```xml
<buses>
  <bus name="my-bus" descriptor="jms://${jms.host}:${jms.port}&username=${jms.user}&password=${jms.password}&jndi=true&jndi_contextfactory=${jms.contextfactory}&jndi_principal=${jms.principal}&jndi_credentials=${jms.credentials}&jndi_connectionfactory=${jms.connectionfactory}">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

The descriptor form is useful when the descriptor is supplied as an external configuration property:

```xml
<buses>
  <bus name="my-bus" descriptor="${myBusDescriptor::loopback://mybus}">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

## Generic JMS Binding Properties

The following properties can be set in the descriptor used to create a JMS bus binding.

| Property                         | Default | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| -------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `jndi`                           | true    | <p>Indicates that JNDI should be used to lookup the connection factory for creating JMS connections.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br>When using JNDI, the address portion of the binding descriptor is used as the address at which to lookup the connection factory. The returned connection factory may connect to a different host.<br></p></div>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `jndi_contextfactory`            | -       | The name of the environment property for specifying the initial context factory to use.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `jndi_principal`                 | -       | The name of the environment property for specifying the identity of the principal for authenticating the caller to the service.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `jndi_credentials`               | -       | The environment property for specifying the credentials of the principal for authenticating the caller to the service.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `jndi_connectionfactory`         | -       | The name of the connection factory to look up in JNDI.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `username`                       | -       | The username to supply in the credentials when opening the JMS connection.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `password`                       | -       | The password to supply in the credentials when opening the JMS connection.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `set_client_id`                  | true    | <p>Specifies whether a client\_id should be set for the connection.<br><br>When true, <code>Connection.setClientID(String)</code> is set on the JMS connection. It is important that the client id be set when using <code>Guaranteed</code> QoS subscriptions as the durable subscriptions issued by the binding are tied to the ClientID.<br><br>When set to true, the client id <code>X-SMA-\<busname>-\<bususer></code> will be used as the JMS ClientID unless <code>use\_legacy\_client\_id</code> or <code>client\_id</code> are set.<br><br>Note that the bususer value used in the ClientID is the username supplied during bus creation to identify the logical bus user, not the <code>username</code> used in the credentials for connecting to the JMS broker.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br>If the JMS connection is created from a JNDI connection factory that provides a pre-configured JMS ClientID, it may cause the JMS provider to throw an exception when the client id is set. Consequently, this value should be set to false if the connection factory is supplying the client id.<br></p></div><p><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br><strong>Breaking Change (3.8)</strong>: Prior to the 3.8 release the ActiveMQ specific provider set a client id of <code>\<username>\<busname></code>. The change to <code>X-SMA-\<busname>-\<bususer></code> is a breaking change for applications that have existing durable subscriptions associated with the old Client ID. The property <code>use\_legacy\_client\_id=true</code> can be set to use the old client id.<br></p></div> |
| `use_legacy_client_id`           | false   | <p>Specifies that the legacy client id should be used.<br><br>When set to true indicates that the default client id of <code>\<bususer>-\<busname></code> should be used rather than the default <code>X-SMA-\<busname>-\<bususer></code>.<br><br>This property is ignored if <code>set\_client\_id</code> is <code>false</code> or <code>client\_id</code> is used to set an explicit JMS ClientID. This property should only be set for applications that were using the activemq bus provider in 3.7 or earlier.<br><br><strong>Since 3.8</strong></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `client_id`                      | -       | <p>Can be used to specify a specific client id for the bus.<br><br>This property can be used to override the default client id when <code>set\_client\_id</code> is set to true.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br>Note that if the bus is shared between multiple applications, the client id should be different for each application, but the same for primary and backup instances.<br></p></div><p><br><br><strong>Since 3.8</strong></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `connection_open_retry_count`    | 10      | <p>Property controlling number of retries to attempt after a failure to open a connection.<br><br>When a connection attempt fails with a reason that isn't known to be a permanent condition, this property controls the number of reconnect attempts to try. Reconnect attempts will be attempted at the interval specified by <code>connection\_open\_retry\_interval</code>.<br><br><strong>Since 3.8</strong></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `connection_open_retry_interval` | 1s      | <p>Property specifying the retry interval for connection open retries in seconds.<br><br>When no time unit suffix is provided for this value it is interpreted as the number of seconds between retries. Otherwise, a time suffix can be provided to qualify the unit as specified in <a href="https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/util/UtlUnit.html#parseDuration(java.lang.String,%20java.util.concurrent.TimeUnit,%20java.util.concurrent.TimeUnit)"><code>UtlUnit.parseDuration(String, TimeUnit, TimeUnit)</code></a>.<br><br>The minimum allowable value for retries is 250ms, specifying a lower value will cause it to be rounded up.<br><br><strong>Since 3.8</strong></p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

## Provider Specific Implementations

### ActiveMQ

While ActiveMQ can be configured via JNDI like any other JMS provider, the platform provides a custom `activemq` bus provider.

The ActiveMQ binding is additionally optimized to:

* Use ActiveMQ's `INDIVIDUAL_ACKNOWLEDGE_MODE` for channels using Guaranteed delivery
* Normalize subscribe and send calls to replace `/` delimited topics with `.` delimited topics which allows the same channel key configuration as other platform bindings

#### Decomposed DDL Format

```xml
<buses>
  <bus name="my-bus">
    <provider>activemq</provider>
    <address>192.168.1.9</address>
    <port>66666</port>
    <properties>
    </properties>
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

#### As Descriptor

```xml
<buses>
  <bus name="my-bus" descriptor="activemq://192.168.1.9:66666">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

#### Binding Specific Properties

The ActiveMQ binding does not have any binding-specific properties beyond the [Generic JMS Binding Properties](#generic-jms-binding-properties) listed above.

### Tibco EMS

**Since 3.8**

While Tibco EMS can be configured via JNDI like any other JMS provider, the platform provides a custom `tibems` bus provider.

The Tibco EMS binding is additionally optimized to:

* Use Tibco EMS' `EXPLICIT_CLIENT_ACKNOWLEDGE` for channels using Guaranteed delivery
* Normalize subscribe and send calls to replace `/` delimited topics with `.` delimited topics which allows the same channel key configuration as other platform bindings

#### Decomposed DDL Format

```xml
<buses>
  <bus name="my-bus">
    <provider>tibems</provider>
    <address>192.168.1.9</address>
    <port>7222</port>
    <properties>
    </properties>
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

#### As Descriptor

```xml
<buses>
  <bus name="my-bus" descriptor="tibems://192.168.1.9:7222">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

#### Binding Specific Properties

The Tibco EMS binding does not have any binding-specific properties beyond the [Generic JMS Binding Properties](#generic-jms-binding-properties) listed above.

## See Also

* [JMS Binding](/talon/concepts-and-architecture/messaging-model/jms-binding) - Conceptual overview
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration


# Loopback Binding

Configuration reference for the Loopback message bus binding.

## Overview

The Loopback binding enables message exchange between applications running in the **same process**. This binding is primarily used for unit testing where several applications may be launched in the same process for easy debugging.

When you configure a bus binding to use a loopback bus, a LoopbackBus instance is statically created on demand using the address provided by the binding. Two applications that connect to a loopback bus with the same name can exchange messages with one another.

For conceptual information about the Loopback binding, see [Loopback Binding](/talon/concepts-and-architecture/messaging-model/loopback-binding).

{% hint style="warning" %}
The Loopback binding **only works within the same process**. It cannot be used for inter-process communication.
{% endhint %}

## Bus Descriptor Format

Loopback buses can be configured using a descriptor string or decomposed DDL format.

### Descriptor String

```
loopback://<busname>&prop1=val1&propN=valN
```

**Example**:

```
loopback://my-bus&topic_starts_with_channel=false&set_bus_and_channel_on_receipt=true
```

### Decomposed DDL Format

```xml
<buses>
  <bus name="my-bus">
    <provider>loopback</provider>
    <address>my-bus</address>
    <properties>
      <topic_starts_with_channel>false</topic_starts_with_channel>
      <set_bus_and_channel_on_receipt>true</set_bus_and_channel_on_receipt>
    </properties>
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

### As Descriptor (Substitution Support)

The descriptor form is useful when the descriptor name may be supplied as an external configuration property that is substituted at runtime, but defaults to loopback for testing:

```xml
<buses>
  <bus name="my-bus" descriptor="${myBusDescriptor::loopback://my-bus&topic_starts_with_channel=false}">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

This example defaults to loopback but can be overridden at runtime with a different bus descriptor (e.g., Solace) via the `myBusDescriptor` property.

## Binding Specific Properties

The Loopback binding does not have any binding-specific properties beyond the [general bus properties](/talon/developing-applications/configuring-messaging/configuring-bus-connections) (such as `topic_starts_with_channel`, `set_bus_and_channel_on_receipt`, etc.) that apply to all message bus bindings.

## Loopback Bus Naming

When working with loopback buses, it is sometimes useful to be able to look up a loopback bus by name. Loopback buses are named using their address (and optionally their port).

If you have configured a loopback bus via `loopback://my-bus`, you can look up the bus as follows:

```java
LoopbackBus bus = LoopbackBus.getInstance("my-bus");
StringBuilder dump = new StringBuilder();
bus.dumpPendingAck(dump);
System.out.println(dump.toString());
```

{% hint style="info" %}
**Port Numbers and the Loopback Bus**

A loopback bus does not require a port to be specified in its configuration. If you do configure a loopback bus with a port, the name of the loopback bus will include the port.

For example, if you specify `loopback://my-bus:80`, the name of the bus will be `my-bus:80`. One consequence of this is that `loopback://my-bus:80` and `loopback://my-bus:90` represent 2 separate buses which cannot communicate with one another.

Other than contributing to the name of the bus, the port has no functional significance.
{% endhint %}

## See Also

* [Loopback Binding](/talon/concepts-and-architecture/messaging-model/loopback-binding) - Conceptual overview
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration


# Executor Binding

Configuration reference for the Executor message bus binding.

## Overview

The Executor Bus Binding is a special bus binding that allows applications to provide send work (via a message) to be processed on a separate thread. The executor binding can be used to perform processor-intensive work in a thread other than an application's main dispatch thread, or can be used as a means to implement outbound gateways in which the executing thread 'pushes' the sent message to an external system.

Work done by the processor of an executor bus is acknowledged and therefore Guaranteed across failures.

For conceptual information about the Executor binding, see [Executor Binding](/talon/concepts-and-architecture/messaging-model/executor-binding).

{% hint style="warning" %}
The Executor Bus is still in incubation and is classified as an experimental feature. The APIs below may change as new features are added to this binding.
{% endhint %}

## Bus Descriptor Format

The Executor binding is configured using decomposed DDL format only.

### Decomposed DDL Format

```xml
<buses>
  <bus name="email-sender">
    <provider>executor</provider>
    <address>audit-logger</address>
    <properties>
      <processor_factory_classname>com.example.EmailGatewayProcessorFactory</processor_factory_classname>
      <from_address>admin@example.com</from_address>
      <smtp_host>mail.example.com</smtp_host>
      <smtp_port>25</smtp_port>
      <smtp_password>admin</smtp_password>
      <!-- ... additional custom properties ... -->
    </properties>
    <channels>
      <channel name="email-alerts">
        <qos>Guaranteed</qos>
      </channel>
    </channels>
  </bus>
</buses>

<apps>
  <app name="email-gateway-app" mainClass="com.example.EmailGateway">
    <messaging>
      <buses>
        <bus name="email-sender">
          <detachedSend enabled="true">
            <queueDrainerCpuAffinityMask>${EMAIL_SENDER_CPU_AFFMASK::0}</queueDrainerCpuAffinityMask>
          </detachedSend>
          <channels>
            <channel name="email-alerts" join="false"/>
          </channels>
        </bus>
      </buses>
    </messaging>
  </app>
</apps>
```

## Executor Bus Properties

An executor bus exposes the following configuration properties:

| Property                                                                                                                                                                        | Default | Description                                                                                                                                                                                                                                                                                                                                                                         |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`processor_factory_classname`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/ExecutorBusConstants.html#PROPNAME_PROCESSOR_PROVIDER_CLASSNAME) | -       | <p>Specifies a factory classname that will be used by the bus to create its processor. The processor factory class must be a subclass of <a href="https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/AbstractExecutorBusProcessorFactory.html"><code>AbstractExecutorBusProcessorFactory</code></a> and expose a zero argument constructor.<br><br></p> |

{% hint style="info" %}
\
\*\*This property is required.\*\*<br>
{% endhint %}

|

### Custom Properties

An application can configure additional properties for the bus that can be used by the bus's processor. In the above example, the bus also defines SMTP-related properties that would be used for sending out e-mails. These custom properties are accessible from the processor via the binding descriptor.

## Threading

The executor bus does not itself create any additional threads; processing of messages is done directly on the thread calling send.

In the case of a Talon application using detached sends is important because otherwise the bus processing would be done on the application's commit thread. Unless the processor implementation has its own thread or thread pool for performing work, it is usually desirable to configure executor buses for detached send.

## Channels

The executor bus will set the channel name on messages it dispatches to the processor. It is good practice for processors that implement their own threading or thread pools to maintain ordered processing on a channel by channel basis.

{% hint style="warning" %}
Attempting to configure an executor bus channel for join will result in a runtime error - an executor bus only supports outbound semantics.
{% endhint %}

### QoS

Provided an executor bus channel is declared as Guaranteed, an AEP Engine will not complete the transaction from which the processing was scheduled until the bus's processor has acknowledged it. This means that if work was sent to the executor bus in response to a received message, the received message won't be acknowledged until the work is done.

If the executor bus channel is defined as BestEffort, the engine will not wait for acknowledgement of work completion before acknowledging received events.

## Address and Port

An executor bus must be configured with an address, but a port is optional. The address may be used by a bus's processor factory as a means of looking up a processor if there are multiple executor buses configured for an application.

## See Also

* [Executor Binding](/talon/concepts-and-architecture/messaging-model/executor-binding) - Conceptual overview
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - General bus configuration
* [`ExecutorBusProcessor`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/ExecutorBusProcessor.html) - Processor interface
* [`AbstractExecutorBusProcessorFactory`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/spi/executor/AbstractExecutorBusProcessorFactory.html) - Factory base class


# Registering Message Interest

The AEP engine creates and manages the lifecycle of the message buses that an application configures for use. When a microservice is configured to join one or more bus channels, the engine issues appropriate topic subscriptions on behalf of the microservice.

<figure><picture><source srcset="/files/oQ3D78uMr24JRTwwRFpn" media="(prefers-color-scheme: dark)"><img src="https://440366835-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FXjW7fsQ5KaRgJ32I4jU7%2Fuploads%2Fgit-blob-9a0d50beac7327092b9d9dacf09000e202b04157%2Ftalon-inbound-message-dispatch-light.png?alt=media" alt=""></picture><figcaption><p>Inbound Message Dispatch</p></figcaption></figure>

The above depicts how messages drawn into the microservice via the issued subscriptions are processed:

* The SMA message bus binding implementation receives a provider specific message from the underlying message bus. The provider specific message is comprised of an application specific data payload and SMA metadata.
* The binding uses an application registered message view factory (generated by ADM) and the SMA message metadata present on the derlying bus message to convert the bus specific message into a message object (called a "Message View" and is also ADM generated).
* The binding uses the SMA message metadata to identify the message channel on which to dispatch the received message.
* The binding wraps the message view in a MessageEvent with a reference to the message channel and dispatches it to the application's AEP Engine, where it is enqueued for processing.
* The AEP Engine picks up the message event and dispatches to each application event handler that has a signature matching the message type.
* Once the AEP Engine stabilizes the results of the application's message processing, a message acknowledgment is dispatched back to the binding.

## Registering Message Interest

For an application to receive messages, it must:

* join the message channels on which the message is sent,
* register message factories for the bus provider to deserialize the message,
* define an EventHandler for the message.

### Configuring Channels For Join <a href="#sendingandreceivingmessages-configuringchannelsforjoin" id="sendingandreceivingmessages-configuringchannelsforjoin"></a>

For the AEP Engine to issue subscriptions for the message channel on which a message is sent, the channel must be joined. Buses and channels are configured via the platform's configuration DDL. The below configuration snippet demonstrates:

* Defining a bus named "sample-bus" with a "new-orders-channel".
* An application that uses the "sample-bus" and joins the "new-orders-channel" (note the usage of `join="true"`)

```xml
<?xml version="1.0"?>
<model xmlns="http://www.neeveresearch.com/schema/x-ddl" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 
  <!-- Define buses and channels that will be shared between applications-->
  <buses>
    <bus name="sample-bus">
      <provider>activemq</provider>
      <address>localhost</address>
      <port>61616</port>
      <channels>
        <channel name="new-orders-channel">
          <qos>Guaranteed</qos>
          <key>NEWORDERS/${Region}/${Department}</key>
        </channel>
      <channels>
    </bus>
  </buses>
   
  <!-- Apps will reference the buses they use in their messaging config -->
  <apps>
    <app name="sample-app" mainClass="com.sample.SampleApp">
      <messaging>
        <factories>
          <factory name="com.sample.messages.OrderMessagesFactory" />
        </factories>
        <buses>
            <bus name="sample-bus">
                <channels>
                    <channel name="new-orders-channel" join="true">
                        <filter>Region=US|Canada</filter>
                    </channel>
                </channels>
            </bus>
        </buses>
      <messaging>
    </app>
  </apps>
</model>
```

#### Channel Filters

A channel filter is used to filter what is received over a message channel. In other words, it is used to determine the subscriptions issued on behalf of the application. See the [Configuring Channels for Join](#sendingandreceivingmessages-configuringchannelsforjoin) section above for an example of using channel filters. In particular, pay attention to the "key" and "filter" elements.

**Channel Filter Syntax**

Channel filter syntax takes the following form:

`var1=val1[|val2][;var2=val3]`

For example, given a channel key of `"NEWORDERS/${Region}/{Department}"`, one can specify a channel filter of `"Region=US|EMEA;Department=Clothing"`. This would join the channel on:

* NEWORDERS/US/Clothing
* NEWORDERS/EMEA/Clothing

If a variable portion of the channel key is omitted in a filter, it will result in the subscription being joined in a wildcard fashion, assuming the underlying bus implementation supports wildcards. So given a channel key of `"NEWORDERS/${Region}/${Department}"` and a channel filter of `"Region=US|EMEA"`, the following subscriptions would be issued during join:

* NEWORDERS/US/\*
* NEWORDERS/EMEA/\*

Finally, if the channel filter is set to null for the channel key in the example above, then the resulting subscription would be:

* NEWORDERS/\*/\*

**Cleaning Channel Filters**

When the global configuration setting nv.sma.cleanchannelfilter is set to true, non-alphanumeric characters in the configured filter values will be replaced by underscores in order to match the keys used on the send side. The below configuration setting can be used to opt-out of this behavior, but typically the default behavior is more desirable:

<table><thead><tr><th width="237.0245361328125">Property</th><th width="108.43524169921875">Default</th><th>Description</th></tr></thead><tbody><tr><td>nv.sma.cleanchannelfilter</td><td>false*</td><td><p>Controls whether or not channel filter values are sanitized by replacing any non-letter or digit character with a '_'. For example, if the channel key is specified as <code>/Orders/${Region}</code> and a filter of <code>Region=Asia/Pac</code> is given, then the filter will match all messages with the resolved key value of <code>/Orders/Asia_Pac</code> (rather than <code>/Orders/Asia/Pac</code>).</p><p><strong>*Default Value:</strong></p><ul><li>In X 3.7, the value defaults to the value specified for <code>nv.sma.cleanmessagekey</code></li><li>In X 3.8 onwards the default value is false.</li></ul></td></tr></tbody></table>

As of the X 3.8, channel filter cleaning has been enhanced to not replace certain wildcard characters that are legal for use in subscriptions.

* **Solace Binding:** A '>' will not be replaced if it represents the whole topic level and a '\*' will not be replaced if it is the last character in the topic level.
* **Loopback Binding:** A filter level of "..." will not be replaced nor will a '\*' found anywhere in the topic level.
* **JMS Binding:** Depends on the provider instance.

Otherwise, any characters that are not alpha-numeric will be replaced. Prior to X 3.8, any non-alphanumeric character was replaced included the wildcard combinations described above.

### Registering Message Factories <a href="#sendingandreceivingmessages-registeringmessagefactories" id="sendingandreceivingmessages-registeringmessagefactories"></a>

A Talon message bus bindings receive messages from the underlying bus in provider specific form but with SMA metadata that is tagged by the corresponding binding in upstream Talon microservices that sent the messages. In order to convert the provider specific message to a message view (object), the receiving binding needs the following:

* The id of the received message
* The id of the received message's factory.
* The factory registered with the Talon SMA runtime.

The first two above are present in the SMA metadata tagged to the inbound message. However, the message factory needs to be registered with the SMA runtime for it to find and use to convert the inbound message to object form. This registration can be done declaratively or programmatically.

#### Declarative Registration <a href="#sendingandreceivingmessages-registrationviaconfigddl" id="sendingandreceivingmessages-registrationviaconfigddl"></a>

Message factories are declaratively registered in the applications DDL configuration as follows:

```xml
<apps>
  <app name="MyApp">
    <messaging>
      <factories>
        <factory name="com.sample.messages.OrderMessagesFactory" />
      </factories>
      <buses>
        <!-- ... -->
      </buses>
    </messaging>
  </app>
</app>
```

#### Programmatic Registration <a href="#sendingandreceivingmessages-addinganeventhandler" id="sendingandreceivingmessages-addinganeventhandler"></a>

Registration can also be done programmatically via the AEP Engine. A common way to do this is to provide an `AppInjectionPoint` for the AEP Engine in the application.

```java
public class MyApp {
   
  @AppInjectionPoint
  public void onEngineCreated(AepEngine engine) {
    engine.registerFactory(new com.example.messages.MyMessageFactory());
    engine.registerFactory(new com.example.messages.other.MyOtherMessageFactory());
  }
}
```

### Adding a Message Handler <a href="#sendingandreceivingmessages-addinganeventhandler" id="sendingandreceivingmessages-addinganeventhandler"></a>

When a message is received by a message bus, it is enqueued into the AEP Engine's inbound event queue for dispatch. A message handler is defined as a method that is annotated with the `@EventHandler` annotation and has a method signature that accepts just a message type (for State Replicated and Event Sourced microservices) or a message type and the entity type of the microservice's store root. The following are examples of these types of handlers

{% tabs %}
{% tab title="Single Parameter Handler" %}

```java
@EventHandler
public void onNewOrder(NewOrderMessage message) {
  /*... do some work ...*/
}
```

{% endtab %}

{% tab title="Two Parameter Handler" %}

```java
@EventHandler
public void onNewOrder(NewOrderMessage message, Repository repository) {
  /*... do some work ...*/
}
```

{% endtab %}
{% endtabs %}

The AEP Engine will ensure that the message is acknowledged once state changes made by the handler have been stabilized. That, coupled with the engine's message deduplication feature, ensures that even in the event of a failover, the handler will be executed once and exactly once in the life of a microservice.

### Preserving Interest on Shutdown <a href="#sendingandreceivingmessages-preservingsubscriptionsonshutdown" id="sendingandreceivingmessages-preservingsubscriptionsonshutdown"></a>

By default, when an AEP Engine is stopped without an error, bus channels that were 'joined' will be 'left', meaning that any subscriptions or interests created by the message bus will be unsubscribed or unregistered. For many applications, it is desirable to preserve subscriptions if an application is being gracefully shutdown for maintenance reasons – one may want messages to be queued for the application while it is down. For such cases the default behavior or unsubscribing on graceful shutdown can be overridden by configuring an application to preserve channel joins on stop:

{% code title="Preserving Interest via configuration" %}

```xml
<app name="sample-app" mainClass="com.sample.SampleApp">
  <messaging>
    ...
  <messaging>
  <preserveChannelJoinsOnStop>true</preserveChannelJoinsOnStop>
</app>
```

{% endcode %}

Note that this property has no effect when an engine shuts down with an error (e.g. [AepEngine.stop(Exception)](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#stop\(java.lang.Exception\)) with a non-null cause. In this case, channel joins are left intact, allowing a backup to take over.

This behavior can also be overridden programmatically on a case by case basis by an EventHandler for the AepEngineStoppingEvent[ setting ](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineStoppingEvent.html)[AepEngineStoppingEvent.setPreserveChannelJoins(boolean)](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepEngineStoppingEvent.html#setPreserveChannelJoins\(boolean\))

{% code title="Removing All Interest Programmatically" %}

```java
@EventHandler
public void onEngineStopping(final AepEngineStoppingEvent event) {
  if(event.getPreserveChannelJoins()) {
    tracer.log("Overriding unsubscribe behavior on application stop to remove subscriptions.");
    event.setPreserveChannelJoins(false);
  }
}
```

{% endcode %}

In the above case, the value set programmatically overrides the configured value for the application.

#### Per Channel Interest Preservation <a href="#sendingandreceivingmessages-perchannelsubscriptionpreservation" id="sendingandreceivingmessages-perchannelsubscriptionpreservation"></a>

Subscription preservation or removal can also be configured more granularly at the channel level. Like the application level configuration setting, this per channel configuration setting only applies to a graceful close.

The following example shows an application configuring subscription preservation on a per channel basis using the [preserveJoinsOnClose](/talon/reference/configuration#channel-subscriptions) configuration property:

**Preserving Per Channel Channel via Configuration**

{% code title="Preserving Interest for a specific channel via configuration" %}

```xml
<app name="sample-app" mainClass="com.sample.SampleApp">
  <preserveChannelJoinsOnStop>true</preserveChannelJoinsOnStop>
  <messaging>
    <buses>
      <bus name="orders-bus">
        <channels>
          <channel name="canceled-orders" join="true">
            <filter>Region=US</filter>
            <preserveJoinsOnClose>Default</preserveJoinsOnClose>
          </channel>
          <channel name="new-orders" join="true">
            <filter>Region=US</filter>
            <preserveJoinsOnClose>Preserve</preserveJoinsOnClose>
          </channel name="app-ping" join="true">
            <filter>Region=US</filter>
            <preserveJoinsOnClose>Leave</preserveJoinsOnClose>
          </channel>
        </channels>
      </bus>
    </buses>
  <messaging>
</app>
```

{% endcode %}

In the above case, if the application is stopped gracefully:

* Subscriptions for the `canceled-orders` channel would be preserved (because `preserveChannelJoinsOnStop=true` at the application level).
* Subscriptions for the `new-orders` channel would be preserved regardless of the configured value for preserveChannelJoinsOnStop.
* Subscriptions for the `app-ping` channel would be unsubscribed regardless of the configured value for preserveChannelJoinsOnStop.

One can also configure per channel subscription preservation programmatically via the message channel's [MessageChannelDescriptor](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageChannelDescriptor.html). A programmatically set value will override that set via DDL configuration, and can be set at any time before the channel is closed:

**Preserving Per Channel Channel Programmatically**

{% code title="Preserving Subscriptions for a specific channel programmatically" %}

```java
@EventHandler(source = "channel4@aeptest1")
public void onChannel4UpEventHandler(final AepChannelUpEvent event) {
  MessageChannel channel = event.getMessageChannel();
  if(channel.getName().equals("app-ping")) {
    channel .getDescriptor().setPreserveJoinsOnClose(PreserveJoinPolicy.Leave);
  }
}
```

{% endcode %}


# Authoring User Code

This section covers how to write the business logic for your Talon microservice. Topics are organized by functional area to help you find the right information quickly.

## Overview

Talon microservices are event-driven applications that process messages within automatic transactions. Your user code defines:

* **What consensus model to use** - Event Sourcing or State Replication
* **How to initialize** - Lifecycle hooks and startup behavior
* **How to process messages** - Business logic in message handlers
* **How to inject messages** - Programmatic and scheduled injection
* **How to expose commands** - Administrative control points
* **How to expose metrics** - Custom telemetry and monitoring

## Getting Started

**New to Talon?** Start here:

1. [Consensus Model](#consensus-model) - Choose and configure your HA policy
2. [Lifecycle](#lifecycle) - Understand initialization and shutdown
3. [Message Processing](#message-processing) - Write your first message handlers
4. Read [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) before writing handlers

## Categories

### Consensus Model

Configure your microservice's high availability and consensus behavior.

* [**Specifying The HA Policy**](/talon/developing-applications/authoring-user-code/consensus-model/specifying-the-ha-policy) - Choose Event Sourcing or State Replication

### Lifecycle

Implement lifecycle methods that the Talon runtime invokes during your microservice's lifecycle.

Lifecycle methods include **accessor methods** (provide data to runtime), **injection methods** (receive runtime objects), and **notification methods** (handle lifecycle events).

* [**Implementing Lifecycle Methods**](/talon/developing-applications/authoring-user-code/lifecycle/implementing-lifecycle-methods) - Implement accessor, injection, and notification methods
* [**Initializing the Microservice**](/talon/developing-applications/authoring-user-code/lifecycle/initializing-the-microservice) - Handle first and initial messages

### Message Processing

Process inbound messages and execute business logic.

* [**Filtering Messages**](/talon/developing-applications/authoring-user-code/message-processing/filtering-messages) - Selectively process messages
* [**Processing Messages**](/talon/developing-applications/authoring-user-code/message-processing/processing-messages) - Core message handling
  * [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Write message handlers
  * [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - Send outbound messages
* [**Unhandled Messages**](/talon/developing-applications/authoring-user-code/message-processing/unhandled-messages) - Handle messages with no registered handler

### Message Injection

Programmatically create and inject messages for processing.

* [**Injecting Messages**](/talon/developing-applications/authoring-user-code/message-injection/injecting-messages) - Inject messages programmatically
* [**Scheduling Messages**](/talon/developing-applications/authoring-user-code/message-injection/scheduling-messages) - Schedule future or periodic messages

### Command and Control

Implement administrative commands for runtime control.

* [**Implementing Command Handlers**](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Create custom administrative commands

### Monitoring

Expose custom statistics and telemetry from your microservice.

* [**Exposing Application Stats**](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Define custom metrics using `@AppStat`

## Related Topics

* [Microservice Template](/talon/developing-applications/microservice-template) - Template-specific guidance for Event Sourcing and State Replication
* [Configuring Messaging](/talon/developing-applications/configuring-messaging) - Set up message bus connections and subscriptions
* [Configuring the Microservice Runtime](/talon/developing-applications/configuring-the-runtime) - Runtime configuration and tuning
* [Concepts & Architecture](/talon/concepts-and-architecture) - Understand how Talon works


# Consensus Model

This section covers how to configure your microservice's consensus model and high availability policy.

## Overview

Talon supports two consensus models for maintaining consistency across replicated microservices:

* **Event Sourcing** - POJO-based stores with event replay for consensus
* **State Replication** - ADM-generated state objects with automated replication

The HA policy determines how your microservice behaves in a cluster and which consensus model it uses.

## Topics

* [**Specifying The HA Policy**](/talon/developing-applications/authoring-user-code/consensus-model/specifying-the-ha-policy) - Configure your microservice's high availability and consensus behavior

## Related Topics

* [Consensus Models](/talon/concepts-and-architecture/consensus-models) - Conceptual overview of Event Sourcing vs State Replication
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How consensus works during message processing
* [Microservice Template](/talon/developing-applications/microservice-template) - Template-specific guidance for each consensus model


# Specifying The HA Policy

The choice of a microservice's HA Policy, aka Consensus Model, is a design choice. Therefore, it is not a configurable option. Instead, the HA policy isspecified via the [AppHAPolicy](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/app/annotations/AppHAPolicy.html) annotation on the microservice's main class.

The following illustrates how to set a microservice's HA policy

```java
import com.neeve.server.app.annotations.*;
 
@AppHaPolicy(StateReplication)
public class MyApp {
     ...
}
```

The [AppHAPolicy](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/app/annotations/AppHAPolicy.html) supports the following values

* StateReplication
* EventSourcing

### Default HA Policy

A microservice that is not annotated with the [AppHAPolicy](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/app/annotations/AppHAPolicy.html) defaults to StateReplication.


# Lifecycle

This section covers how to implement lifecycle methods that control your microservice's initialization and shutdown behavior.

## Overview

Talon microservices have a well-defined lifecycle from initialization through running to shutdown. You can hook into this lifecycle using annotations to perform setup, initialization, and cleanup tasks.

## Lifecycle Phases

1. **Construction** - Class instantiation
2. **Initialization** - `@AppInit` methods execute
3. **Running** - Message processing begins
4. **Shutdown** - `@AppShutdown` methods execute

## Topics

* [**Implementing Lifecycle Methods**](/talon/developing-applications/authoring-user-code/lifecycle/implementing-lifecycle-methods) - Use `@AppInit` and `@AppShutdown` annotations
* [**Initializing the Microservice**](/talon/developing-applications/authoring-user-code/lifecycle/initializing-the-microservice) - Handle first and initial messages for state initialization

## Related Topics

* [Application Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Conceptual overview of the full lifecycle


# Implementing Lifecycle Methods

## Overview

Lifecycle methods allow you to hook into the Talon microservice lifecycle at specific points during startup, operation, and shutdown. The Talon runtime invokes these methods at well-defined stages, enabling you to:

* Receive platform objects via dependency injection
* Provide application objects for annotation scanning
* Initialize resources and state
* React to lifecycle transitions
* Clean up resources during shutdown

Lifecycle methods fall into three categories:

### Injection Methods

The Talon runtime invokes injection methods to provide your application with handles to platform objects. These methods are marked with `@AppInjectionPoint` and allow the runtime to inject objects like the AEP engine, message sender, and configuration descriptors.

### Accessor Methods

The Talon runtime invokes accessor methods to gather application objects containing various annotations. These methods allow you to organize your code across multiple classes while making them discoverable to the Talon runtime.

### Notification Methods

The Talon runtime invokes notification methods to notify your application of lifecycle transitions and operational events. These include initialization, finalization, and lifecycle event handlers.

**See Also**: [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Complete lifecycle flow and timing

***

## Dependency Injection

Use the `@AppInjectionPoint` annotation to receive platform objects from the Talon runtime.

### Injectable Types

The Talon runtime can inject the following types:

| Type                    | Purpose                                                 | Injection Timing              |
| ----------------------- | ------------------------------------------------------- | ----------------------------- |
| **SrvAppLoader**        | Access to application and XVM configuration descriptors | Early in Open phase           |
| **AepEngineDescriptor** | Modify engine configuration before creation             | After HA policy determination |
| **AepMessageSender**    | Send outbound messages                                  | After engine creation         |
| **AepEngine**           | Access to running engine instance                       | After engine creation         |

### Injection on Fields

You can inject directly into fields:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
    @AppInjectionPoint
    private AepMessageSender sender;

    @AppInjectionPoint
    private AepEngine engine;

    @EventHandler
    public void onOrder(NewOrderMessage message) {
        // Use sender to send outbound messages
        OrderConfirmation confirmation = OrderConfirmation.create();
        confirmation.setOrderId(message.getOrderId());
        sender.sendMessage("order-confirmations", confirmation);
    }
}
```

### Injection on Methods

Alternatively, inject via setter methods for more control:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
    private AepEngine engine;
    private AepMessageSender sender;

    @AppInjectionPoint
    public void setEngine(AepEngine engine) {
        this.engine = engine;
        logger.info("Engine injected: {}", engine.getName());
    }

    @AppInjectionPoint
    public void setSender(AepMessageSender sender) {
        this.sender = sender;
    }
}
```

### Injecting the Application Loader

The application loader provides access to configuration descriptors:

```java
@AppInjectionPoint
public void setLoader(SrvAppLoader loader) {
    // Access XVM configuration
    XRuntime runtime = loader.getXRuntime();

    // Access application descriptor
    SrvAppDescriptor appDescriptor = loader.getAppDescriptor();

    // Access XVM descriptor
    SrvXvmDescriptor xvmDescriptor = loader.getXvmDescriptor();
}
```

### Modifying the Engine Descriptor

Inject the engine descriptor to programmatically configure the engine before it's created:

```java
@AppInjectionPoint
public void setEngineDescriptor(AepEngineDescriptor descriptor) {
    // Configure threading
    descriptor.setInputDisruptorCount(2);

    // Configure adaptive batching
    descriptor.setAdaptiveBatchingEnabled(true);
    descriptor.setAdaptiveBatchingCeiling(100);

    // Configure transaction log
    descriptor.getTransactionLogDescriptor().setEnabled(true);
}
```

{% hint style="info" %}
**Timing**: Engine descriptor injection occurs after the `@AppHAPolicy` is read but before the engine is created. Any HA policy set programmatically in the descriptor will override the annotation value.
{% endhint %}

**See Also**: [AepEngineDescriptor Javadoc](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngineDescriptor.html)

***

## Accessor Methods

Accessor methods allow you to provide objects to the Talon runtime for annotation scanning. This enables you to organize your code across multiple classes while keeping handlers, stats, and configuration discoverable.

### Unified Accessor - @AppIntrospectionPoints

Use `@AppIntrospectionPoints` to provide objects for all types of annotation scanning:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
    private OrderHandler orderHandler = new OrderHandler();
    private MetricsCollector metricsCollector = new MetricsCollector();

    @AppIntrospectionPoints
    public void getIntrospectionPoints(Set<Object> objects) {
        objects.add(orderHandler);
        objects.add(metricsCollector);
    }
}

class OrderHandler {
    @AppInjectionPoint
    private AepMessageSender sender;

    @EventHandler
    public void onNewOrder(NewOrderMessage message) {
        // Process order
    }

    @Command(name = "cancelOrder", description = "Cancel an order")
    public String cancelOrder(@Argument(name = "orderId", position = 1) long orderId) {
        // Cancel order
        return "Order " + orderId + " cancelled";
    }
}

class MetricsCollector {
    @AppStat(name = "Orders Processed")
    private volatile long ordersProcessed = 0;

    @Configured(property = "myapp.metricsEnabled", defaultValue = "true")
    private boolean metricsEnabled;
}
```

{% hint style="info" %}
**When to Use**: @AppIntrospectionPoints is called multiple times during the lifecycle (before configuration injection, command handler discovery, event handler discovery, etc.). This allows you to construct objects that depend on earlier lifecycle steps.
{% endhint %}

### Fine-Grained Accessors

For large applications, use fine-grained accessors to reduce the number of objects scanned for each annotation type:

#### @AppConfiguredAccessor

Provide objects containing `@Configured` annotations:

```java
@AppConfiguredAccessor
public void getConfiguredObjects(Set<Object> objects) {
    objects.add(configuredComponent);
}
```

#### @AppEventHandlerContainersAccessor

Provide objects containing `@EventHandler` methods:

```java
@AppEventHandlerContainersAccessor
public void getEventHandlers(Set<Object> containers) {
    containers.add(orderHandler);
    containers.add(paymentHandler);
}
```

#### @AppCommandHandlerContainersAccessor

Provide objects containing `@Command` methods:

```java
@AppCommandHandlerContainersAccessor
public void getCommandHandlers(Set<Object> containers) {
    containers.add(adminCommands);
}
```

#### @AppStatContainersAccessor

Provide objects containing `@AppStat` annotations:

```java
@AppStatContainersAccessor
public void getStatContainers(Set<Object> containers) {
    containers.add(metricsCollector);
}
```

### Programmatic Event Handler

For advanced use cases, provide a programmatic event handler instead of using annotations:

```java
@AppEventHandlerAccessor
public IEventHandler getDefaultEventHandler() {
    return new IEventHandler() {
        @Override
        public void onEvent(IEvent event) {
            // Handle all events programmatically
        }

        @Override
        public void onMessage(MessageView message, MessageView state) {
            // Handle all messages programmatically
        }
    };
}
```

{% hint style="warning" %}
**Note**: The default event handler is only invoked for events/messages not handled by `@EventHandler` annotated methods (unless configured with `DefaultHandlerDispatchPolicy=DispatchAlways`).
{% endhint %}

**See Also**:

* [Annotations Reference](/talon/reference/annotations) - Complete annotation details
* [Injecting Configuration](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) - Using @Configured
* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Using @Command
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Using @AppStat

***

## Initialization and Finalization

### Application Initialization - @AppInitializer

The `@AppInitializer` method is invoked after the engine is created and injected but before it is started:

```java
@AppInitializer
public void initialize() {
    logger.info("Initializing application");

    // Load reference data
    loadReferenceData();

    // Initialize connection pools
    initializeConnectionPools();

    // Set up internal data structures
    setupDataStructures();
}
```

**What's Available at Initialization**:

* ✅ Injected platform objects (engine, sender, etc.)
* ✅ Configuration values
* ✅ Application state factory has been provided
* ❌ Engine is not yet started
* ❌ Messaging is not yet connected
* ❌ Store is not yet open

**Thread Safety**:

{% hint style="danger" %}
**Do not access or modify microservice state** in the initializer for State Replication applications. The store is not yet open and state objects are not yet available. Use lifecycle events (like `AepEngineActiveEvent`) for state-dependent initialization.
{% endhint %}

### Application Finalization - @AppFinalizer

The `@AppFinalizer` method is invoked during shutdown after the engine is stopped:

```java
@AppFinalizer
public void finalize() {
    logger.info("Finalizing application");

    // Close external connections
    closeConnections();

    // Release resources
    releaseResources();

    // Flush logs or metrics
    flushMetrics();
}
```

**What's Available at Finalization**:

* ✅ Injected platform objects still available
* ❌ Engine is stopped
* ❌ Messaging is disconnected
* ❌ Store is closed

**See Also**: [Lifecycle - Initialize Application](/talon/concepts-and-architecture/microservice-operation/lifecycle#initialize-application)

***

## State Factory for State Replication

For State Replication microservices, provide the state factory via `@AppStateFactoryAccessor`:

```java
@AppHAPolicy(HAPolicy.StateReplication)
public class MyApp {
    @AppStateFactoryAccessor
    public IAepApplicationStateFactory getStateFactory() {
        return new IAepApplicationStateFactory() {
            @Override
            public Repository createState(MessageView view) {
                return Repository.create();
            }
        };
    }

    @EventHandler
    public void onCreateCustomer(CreateCustomerMessage message, Repository repository) {
        Customer customer = Customer.create();
        customer.setProfile(message.getCustomerProfile().copy());
        repository.getCustomers().put(message.getCustomerId(), customer);
    }
}
```

**See Also**: [State Replication Template](/talon/developing-applications/microservice-template/state-replication-template)

***

## Synchronous Applications

### Main Method - @AppMain

For microservices that are not event-driven (sender-only applications), implement a main method:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class SenderApp {
    @AppInjectionPoint
    private AepMessageSender sender;

    @Configured(property = "sender.messageCount", defaultValue = "1000")
    private int messageCount;

    @Configured(property = "sender.ratePerSecond", defaultValue = "100")
    private int ratePerSecond;

    @AppMain
    public void main() {
        logger.info("Starting sender with {} messages at {} msgs/sec",
            messageCount, ratePerSecond);

        RateLimiter rateLimiter = RateLimiter.create(ratePerSecond);

        for (int i = 0; i < messageCount; i++) {
            rateLimiter.acquire();

            TestMessage message = TestMessage.create();
            message.setSequenceNumber(i);
            message.setTimestamp(System.currentTimeMillis());

            sender.sendMessage("test-messages", message);
        }

        logger.info("Sending complete");
    }
}
```

**Execution**:

* Invoked in a separate thread after the engine becomes primary
* Runs asynchronously from message processing
* Suitable for driving application logic without inbound messages

{% hint style="info" %}
**When to Use**: Use `@AppMain` for applications that generate messages or drive their own operation synchronously rather than reacting to inbound messages.
{% endhint %}

**See Also**: [Lifecycle - Synchronous Applications](/talon/concepts-and-architecture/microservice-operation/lifecycle#synchronous-applications)

***

## Handling Lifecycle Events

Use `@EventHandler` to react to lifecycle events dispatched by the Talon runtime.

### Engine Lifecycle Events

#### AepEngineCreatedEvent

Dispatched after the engine is created but before it is started:

```java
@EventHandler
public void onEngineCreated(AepEngineCreatedEvent event) {
    logger.info("Engine created: {}", event.getEngine().getName());
    // Engine configuration is finalized
    // Messaging not yet started
}
```

#### AepMessagingPrestartEvent

Dispatched before the engine attempts to connect to messaging:

```java
@EventHandler
public void onMessagingPrestart(AepMessagingPrestartEvent event) {
    logger.info("About to start messaging");
    // Last chance before messaging connections are established
}
```

#### AepEngineStartedEvent

Dispatched after the engine determines its role (primary or backup):

```java
@EventHandler
public void onEngineStarted(AepEngineStartedEvent event) {
    boolean isPrimary = event.getEngine().isPrimary();
    logger.info("Engine started as {}", isPrimary ? "primary" : "backup");
}
```

#### AepEngineActiveEvent

Dispatched when the engine becomes the active primary:

```java
@EventHandler
public void onEngineActive(AepEngineActiveEvent event) {
    logger.info("Engine is now primary and active");

    // Safe to start background processing
    startBackgroundTasks();

    // Safe to schedule periodic tasks
    scheduler.scheduleAtFixedRate(this::generateReports,
        0, 1, TimeUnit.HOURS);
}
```

{% hint style="success" %}
**Best Practice**: Use `AepEngineActiveEvent` to start background threads, scheduled tasks, or any processing that should only run on the primary instance.
{% endhint %}

#### AepMessagingStartedEvent

Dispatched after messaging startup completes:

```java
@EventHandler
public void onMessagingStarted(AepMessagingStartedEvent event) {
    logger.info("Messaging started successfully");
    // All configured bindings have been attempted
}
```

#### AepEngineStoppedEvent

Dispatched when the engine stops:

```java
@EventHandler
public void onEngineStopped(AepEngineStoppedEvent event) {
    logger.info("Engine stopped");
    // Perform any final cleanup
}
```

### Messaging Lifecycle Events

#### AepChannelUpEvent

Dispatched when a channel connection is established:

```java
@EventHandler
public void onChannelUp(AepChannelUpEvent event) {
    logger.info("Channel up: {}", event.getChannel().getName());
    // Channel is ready to send/receive messages
}
```

#### AepChannelDownEvent

Dispatched when a channel connection is lost:

```java
@EventHandler
public void onChannelDown(AepChannelDownEvent event) {
    logger.warn("Channel down: {}", event.getChannel().getName());
    // Channel is no longer available
}
```

#### AepBusBindingUpEvent

Dispatched when a bus binding is fully operational:

```java
@EventHandler
public void onBindingUp(AepBusBindingUpEvent event) {
    logger.info("Binding up: {}", event.getBinding().getName());
    // All channels for this binding are ready
}
```

### Store Lifecycle Events (State Replication)

#### IStoreMemberUpEvent

Dispatched when a new member joins the store cluster:

```java
@EventHandler
public void onStoreMemberUp(IStoreMemberUpEvent event) {
    logger.info("Store member up: {}", event.getMember().getId());
}
```

#### IStoreMemberInitCompleteEvent

Dispatched when store initialization completes:

```java
@EventHandler
public void onStoreMemberInitComplete(IStoreMemberInitCompleteEvent event) {
    logger.info("Store initialization complete");
    // State is now available for access
    // Safe to perform state-dependent initialization
}
```

#### IStoreBindingRoleChangedEvent

Dispatched when the store binding's role changes:

```java
@EventHandler
public void onStoreRoleChanged(IStoreBindingRoleChangedEvent event) {
    logger.info("Store role changing from {} to {}",
        event.getOldRole(), event.getNewRole());
}
```

### Alert Events

Alert events signal exceptional conditions:

#### AepMessagingFailedEvent

```java
@EventHandler
public void onMessagingFailed(AepMessagingFailedEvent event) {
    logger.error("Messaging failed", event.getCause());
    // Messaging has shut down due to failure
    // Consider alerting operations
}
```

#### AepBusBindingDownEvent

```java
@EventHandler
public void onBindingDown(AepBusBindingDownEvent event) {
    logger.error("Binding down: {}", event.getBinding().getName());
    // Binding has failed
    // Messages may be lost until binding recovers
}
```

**See Also**: [Events Reference](/talon/reference/events) - Complete event documentation

***

## Complete Example

Here's a complete microservice demonstrating lifecycle method implementation:

```java
package com.example.orderprocessor;

import com.neeve.aep.AepEngine;
import com.neeve.aep.AepEngineDescriptor;
import com.neeve.aep.AepMessageSender;
import com.neeve.aep.IAepApplicationStateFactory;
import com.neeve.aep.annotations.EventHandler;
import com.neeve.aep.event.AepEngineActiveEvent;
import com.neeve.aep.event.AepEngineCreatedEvent;
import com.neeve.cli.annotations.Configured;
import com.neeve.server.app.SrvAppLoader;
import com.neeve.server.app.annotations.*;
import com.neeve.sma.MessageView;

import java.util.Set;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

@AppHAPolicy(HAPolicy.StateReplication)
public class OrderProcessorApp {

    // Injected platform objects
    private AepEngine engine;
    private AepMessageSender sender;

    // Configuration
    @Configured(property = "orderprocessor.maxOrderSize", defaultValue = "1000000")
    private int maxOrderSize;

    // Statistics
    @AppStat(name = "Orders Processed")
    private volatile long ordersProcessed = 0;

    // Background processing
    private ScheduledExecutorService scheduler;

    // Additional components
    private OrderValidator orderValidator = new OrderValidator();
    private MetricsCollector metricsCollector = new MetricsCollector();

    /**
     * Inject the application loader for access to configuration
     */
    @AppInjectionPoint
    public void setLoader(SrvAppLoader loader) {
        logger.info("Application loader injected");
    }

    /**
     * Inject and configure the engine descriptor before engine creation
     */
    @AppInjectionPoint
    public void setEngineDescriptor(AepEngineDescriptor descriptor) {
        // Enable adaptive batching for better throughput
        descriptor.setAdaptiveBatchingEnabled(true);
        descriptor.setAdaptiveBatchingCeiling(100);

        logger.info("Engine descriptor configured");
    }

    /**
     * Inject the engine after creation
     */
    @AppInjectionPoint
    public void setEngine(AepEngine engine) {
        this.engine = engine;
        logger.info("Engine injected: {}", engine.getName());
    }

    /**
     * Inject the message sender
     */
    @AppInjectionPoint
    public void setSender(AepMessageSender sender) {
        this.sender = sender;
        logger.info("Message sender injected");
    }

    /**
     * Provide additional objects for annotation scanning
     */
    @AppIntrospectionPoints
    public void getIntrospectionPoints(Set<Object> objects) {
        objects.add(orderValidator);
        objects.add(metricsCollector);
    }

    /**
     * Provide the state factory for State Replication
     */
    @AppStateFactoryAccessor
    public IAepApplicationStateFactory getStateFactory() {
        return new IAepApplicationStateFactory() {
            @Override
            public OrderRepository createState(MessageView view) {
                return OrderRepository.create();
            }
        };
    }

    /**
     * Initialize the application after engine creation
     */
    @AppInitializer
    public void initialize() {
        logger.info("Initializing application with maxOrderSize={}", maxOrderSize);

        // Initialize components
        orderValidator.initialize(maxOrderSize);

        // Create scheduler for background tasks (but don't start tasks yet)
        scheduler = Executors.newScheduledThreadPool(1);

        logger.info("Application initialized");
    }

    /**
     * Handle engine creation
     */
    @EventHandler
    public void onEngineCreated(AepEngineCreatedEvent event) {
        logger.info("Engine created, preparing for startup");
    }

    /**
     * Handle engine becoming active - safe to start background processing
     */
    @EventHandler
    public void onEngineActive(AepEngineActiveEvent event) {
        logger.info("Engine is now active as primary");

        // Start background tasks only on the primary
        scheduler.scheduleAtFixedRate(() -> {
            logger.info("Orders processed: {}", ordersProcessed);
            metricsCollector.recordOrderCount(ordersProcessed);
        }, 60, 60, TimeUnit.SECONDS);

        logger.info("Background tasks started");
    }

    /**
     * Handle inbound order messages
     */
    @EventHandler
    public void onNewOrder(NewOrderMessage message, OrderRepository repository) {
        // Validate order
        if (!orderValidator.validate(message)) {
            logger.warn("Invalid order rejected: {}", message.getOrderId());
            sendRejection(message, "Validation failed");
            return;
        }

        // Create order entity
        Order order = Order.create();
        order.setOrderId(message.getOrderId());
        order.setCustomerId(message.getCustomerId());
        order.setAmount(message.getAmount());
        order.setStatus(OrderStatus.PENDING);

        // Store in repository
        repository.getOrders().put(message.getOrderId(), order);

        // Send confirmation
        OrderConfirmation confirmation = OrderConfirmation.create();
        confirmation.setOrderId(message.getOrderId());
        confirmation.setStatus("ACCEPTED");
        sender.sendMessage("order-confirmations", confirmation);

        // Update stats
        ordersProcessed++;
    }

    /**
     * Handle order cancellations
     */
    @EventHandler
    public void onCancelOrder(CancelOrderMessage message, OrderRepository repository) {
        Order order = repository.getOrders().get(message.getOrderId());
        if (order != null) {
            order.setStatus(OrderStatus.CANCELLED);
            logger.info("Order {} cancelled", message.getOrderId());
        }
    }

    /**
     * Send order rejection
     */
    private void sendRejection(NewOrderMessage message, String reason) {
        OrderRejection rejection = OrderRejection.create();
        rejection.setOrderId(message.getOrderId());
        rejection.setReason(reason);
        sender.sendMessage("order-rejections", rejection);
    }

    /**
     * Clean up resources on shutdown
     */
    @AppFinalizer
    public void finalize() {
        logger.info("Finalizing application");

        // Shut down background tasks
        if (scheduler != null) {
            scheduler.shutdown();
            try {
                if (!scheduler.awaitTermination(5, TimeUnit.SECONDS)) {
                    scheduler.shutdownNow();
                }
            } catch (InterruptedException e) {
                scheduler.shutdownNow();
            }
        }

        // Final metrics flush
        metricsCollector.flush();

        logger.info("Application finalized, processed {} orders total", ordersProcessed);
    }
}

/**
 * Order validator component with configuration
 */
class OrderValidator {
    private int maxOrderSize;

    public void initialize(int maxOrderSize) {
        this.maxOrderSize = maxOrderSize;
    }

    public boolean validate(NewOrderMessage message) {
        return message.getAmount() > 0 &&
               message.getAmount() <= maxOrderSize &&
               message.getCustomerId() > 0;
    }
}

/**
 * Metrics collector component with stats
 */
class MetricsCollector {
    @AppStat(name = "Peak Orders Per Minute")
    private volatile long peakOrdersPerMinute = 0;

    public void recordOrderCount(long count) {
        // Update peak if needed
        if (count > peakOrdersPerMinute) {
            peakOrdersPerMinute = count;
        }
    }

    public void flush() {
        // Flush any pending metrics
    }
}
```

***

## Lifecycle Execution Order

Lifecycle methods are invoked in the following order during microservice startup:

| Step | Annotation/Method                                  | Purpose                                    |
| ---- | -------------------------------------------------- | ------------------------------------------ |
| 1    | Load main class                                    | XVM loads the microservice                 |
| 2    | `@AppInjectionPoint` (SrvAppLoader)                | Inject application loader                  |
| 3    | `@AppHAPolicy`                                     | Read HA policy from annotation             |
| 4    | `@AppInjectionPoint` (AepEngineDescriptor)         | Inject engine descriptor for configuration |
| 5    | `@AppConfiguredAccessor`                           | Get objects for configuration injection    |
| 6    | `@Configured`                                      | Inject configuration values                |
| 7    | `@AppCommandHandlerContainersAccessor`             | Get command handler containers             |
| 8    | `@AppStatContainersAccessor`                       | Get stat containers                        |
| 9    | `@AppEventHandlerContainersAccessor`               | Get event handler containers               |
| 10   | `@AppStateFactoryAccessor`                         | Get state factory (State Replication)      |
| 11   | Create engine                                      | Engine is instantiated                     |
| 12   | `AepEngineCreatedEvent`                            | Engine created event dispatched            |
| 13   | `@AppInjectionPoint` (AepEngine, AepMessageSender) | Inject engine and sender                   |
| 14   | `@AppInitializer`                                  | Initialize application                     |
| 15   | Start engine                                       | Engine determines role and starts          |
| 16   | `AepEngineStartedEvent`                            | Engine started event dispatched            |
| 17   | `AepEngineActiveEvent`                             | Engine active event (if primary)           |
| 18   | `@AppMain`                                         | Main method invoked (if present)           |

During shutdown:

| Step | Annotation/Method       | Purpose                         |
| ---- | ----------------------- | ------------------------------- |
| 1    | Stop engine             | Engine stops processing         |
| 2    | `AepEngineStoppedEvent` | Engine stopped event dispatched |
| 3    | `@AppFinalizer`         | Finalize application            |

**See Also**: [Lifecycle - Complete Flow](/talon/concepts-and-architecture/microservice-operation/lifecycle)

***

## Best Practices

### ✅ Do

1. **Use AepEngineActiveEvent for primary-only initialization**

   ```java
   @EventHandler
   public void onEngineActive(AepEngineActiveEvent event) {
       // Start background tasks only on primary
       startBackgroundProcessing();
   }
   ```
2. **Use @AppIntrospectionPoints for simple applications**
   * Reduces boilerplate for small to medium applications
   * Single method to provide all objects
3. **Use fine-grained accessors for large applications**
   * Improves startup performance by reducing object scanning
   * Better organization of handler containers
4. **Inject AepEngineDescriptor to configure the engine**

   ```java
   @AppInjectionPoint
   public void setEngineDescriptor(AepEngineDescriptor descriptor) {
       descriptor.setAdaptiveBatchingEnabled(true);
   }
   ```
5. **Use field injection for simplicity**
   * Less code
   * Clear and concise
6. **Clean up resources in @AppFinalizer**

   ```java
   @AppFinalizer
   public void finalize() {
       closeConnections();
       releaseResources();
   }
   ```

### ❌ Don't

1. **Don't access microservice state in @AppInitializer (State Replication)**

   ```java
   @AppInitializer
   public void initialize() {
       // ❌ WRONG: Store not yet open
       // repository.getCustomers().put(1L, customer);

       // ✅ CORRECT: Use AepEngineActiveEvent or message handler
   }
   ```
2. **Don't start background tasks in @AppInitializer**

   ```java
   @AppInitializer
   public void initialize() {
       // ❌ WRONG: Will run on both primary and backup
       // scheduler.scheduleAtFixedRate(task, 0, 1, TimeUnit.MINUTES);

       // ✅ CORRECT: Start in AepEngineActiveEvent (primary only)
   }
   ```
3. **Don't perform long-running operations in lifecycle methods**
   * Keep initialization fast
   * Use background threads for heavy operations
4. **Don't assume messaging is ready in @AppInitializer**

   ```java
   @AppInitializer
   public void initialize() {
       // ❌ WRONG: Messaging not yet started
       // sender.sendMessage("init-complete", message);

       // ✅ CORRECT: Send in AepEngineActiveEvent
   }
   ```
5. **Don't use @AppMain for event-driven applications**
   * @AppMain is for sender-only applications
   * Event-driven apps should use @EventHandler

### Thread Safety

* **Lifecycle methods are single-threaded**: Only one lifecycle method executes at a time
* **Event handlers are single-threaded**: Message processing is single-threaded by design
* **Background threads require synchronization**: If you start background threads, ensure proper synchronization when accessing shared state
* **Volatile fields for stats**: Use volatile for statistics accessed from background threads

```java
// ✅ CORRECT: Volatile for cross-thread access
@AppStat(name = "Orders Processed")
private volatile long ordersProcessed = 0;

// ✅ CORRECT: Synchronized access from background thread
scheduler.scheduleAtFixedRate(() -> {
    long count = ordersProcessed; // Safe: volatile read
    logger.info("Orders: {}", count);
}, 60, 60, TimeUnit.SECONDS);
```

***

## See Also

### Conceptual

* [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Complete lifecycle flow and timing
* [Development Model](/talon/concepts-and-architecture/microservice-architecture/development-model) - Lifecycle method categories

### Reference

* [Annotations](/talon/reference/annotations) - Complete annotation reference
* [Events](/talon/reference/events) - Lifecycle and runtime events

### How-To Guides

* [Injecting Configuration](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) - Using @Configured
* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Using @Command
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Using @AppStat
* [Initializing the Microservice](/talon/developing-applications/authoring-user-code/lifecycle/initializing-the-microservice) - Initialization patterns

### Templates

* [Event Sourcing Template](/talon/developing-applications/microservice-template/event-sourcing-template) - Event Sourcing example
* [State Replication Template](/talon/developing-applications/microservice-template/state-replication-template) - State Replication example


# Initializing the Microservice

Microservice initialization refers to the activities that must be performed before live message processing begins. Because these activities may read from or update the microservice store, they are carried out by injecting *initial messages* into the message stream. The initialization logic is then executed within the message handler(s) for these initial messages.

Initialization messages are specified in the `AepMessagingPrestartEvent`event handler. This event handler is invoked just before the AEP engine starts messaging for its microservice.

## Specifying the First Initialization Message

Perform the following steps, as illustrated below, to specify the first initialization message:

* Create an `AepMessagingPrestartEvent` event handler
* In the event handler, set the first message in this supplied `AepMessagingPrestartEvent`event
* Create a message handler for the first initialization message.
* Perform the initialization activities in the first message handler

```java
@EventHandler
public void onMessagingPrestart(AepMessagingPrestartEvent event) {
  event.setFirstMessage(MyFirstMessage.create());
}

@EventHandler
public void onFirstMessage(MyFirstMessage message) {
  /* do something */
}
```

## Specifying Additional Initialization Messages

In large applications composed of numerous loosely coupled event handlers, it can be beneficial to be able to register multiple initialization message handlers. The code illustrated below supplies two additional initialization messages that are used to initialize distinct code components.

```java
public class ServiceMain {
  @EventHandler
  public void onMessagingPrestart(AepMessagingPrestartEvent event) {
    event.setFirstMessage(MyFirstMessage.create());
    event.addInitialMessage(Component1InitialMessage.create());
    event.addInitialMessage(Component2InitialMessage.create());
  }
  
  @EventHandler
  public void onFirstMessage(MyFirstMessage message) {
    /* do something */
  }
}

public class ServiceComponent1 {
  @EventHandler
  public void onFirstMessage(Component1InitialMessage message) {
    /* do something */
  }
}
 
public class ServiceComponent2 {
  @EventHandler
  public void onFirstMessage(Component2InitialMessage message) {
    /*do something else*/
  }
}
```

## HA Implications for Initial Messages

Technically speaking, microservice initialization is a one-time activity performed only when a microservice instance is started for the very first time - from an empty persistent store. In contrast, the messaging prestart event -`AepMessagingPrestartEvent` - is dispatched just before a microservice instance establishes a connection to the messaging bus. This event occurs **each time an instance is elected as primary**, which can happen multiple times during the lifecycle of a microservice when any of the following occurs:

* The first instance in the microservice cluster is started
* A backup instance takes over after the primary instance fails.

This distinction is important because **initialization messages can be dispatched multiple times** over the lifetime of a microservice - each time a new primary is elected. However, **regardless of how an instance becomes primary**, its state has already been initialized *before* it establishes a messaging connection. This leads to an important consideration for all instances *except* the very first one (the one that initializes from an empty store):

* For **Event Sourced** Microservices

  * State is initialized by replaying inbound messages, including the *initial messages*.
  * These initial messages are:
    * Processed once during replay (as part of state initialization), and
      * Processed again after messaging starts, when they are re-received.

  The microservice logic must handle this duplication correctly.
* For **State Replication** Microservices:
  * Any changes made by initial message processing are already present in the replicated state.
  * When these messages are reprocessed after messaging starts, the logic must **ensure consistency** and avoid redundant state changes.

In both models, the handling of initial messages must account for prior state initialization to ensure correctness and consistency.


# Message Processing

This section covers how to process messages in your microservice handlers, including filtering, handling, and dealing with unhandled messages.

## Overview

Message processing is the core of Talon microservice development. Messages arrive on subscribed channels, are filtered and dispatched to handlers, where business logic executes within automatic transactions.

## Message Processing Flow

1. **Message Arrival** - Inbound message arrives on subscribed channel
2. **Duplicate Detection** - Sequence numbers checked to detect and discard duplicates
3. **Filtering** - Optional filter determines if message should be processed
4. **Handler Dispatch** - AEP Engine dispatches to handler based on message type
5. **Business Logic** - Handler reads message, updates store, creates outbound messages
6. **Transaction Commit** - Handler returns, transaction commits with consensus
7. **Message Acknowledgment** - Inbound message acknowledged

## Topics

* [**Detecting Duplicates**](/talon/developing-applications/authoring-user-code/message-processing/detecting-duplicates) - Detect and discard duplicate messages using sequence numbers
* [**Filtering Messages**](/talon/developing-applications/authoring-user-code/message-processing/filtering-messages) - Use message filters to selectively process messages
* [**Processing Messages**](/talon/developing-applications/authoring-user-code/message-processing/processing-messages) - Core message handling and business logic
  * [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Write message handlers
  * [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - Send outbound messages
* [**Unhandled Messages**](/talon/developing-applications/authoring-user-code/message-processing/unhandled-messages) - Handle messages with no registered handler

## Related Topics

* [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) - Conceptual overview
* [Transactions](/talon/concepts-and-architecture/transactions) - How transactions work
* [Registering Message Interest](/talon/developing-applications/configuring-messaging/registering-message-interest) - Subscribe to message channels


# Detecting Duplicates

The AEP Engine uses sequence numbers from inbound messages to detect and discard duplicate messages before they are dispatched to application handlers. This ensures that applications do not process the same message multiple times.

## Overview

When an upstream microservice sends messages with sequence numbers enabled, the receiving AEP Engine can use those sequence numbers to identify duplicates. This is particularly important in scenarios involving:

* Message bus retransmissions
* Failover and recovery
* Network issues that cause duplicate delivery

By default, duplicate detection is enabled. If an AEP Engine receives a message with a sequence number greater than 1 that is less than or equal to a previously received message's sequence number (for the same bus+channel+qos combination), the message is considered a duplicate and is **not** dispatched to the application.

## How It Works

Duplicate detection operates on the inbound message processing pipeline:

1. Message arrives from message bus
2. **Duplicate Detection** - Sequence number is checked against previously received messages
3. If duplicate: Message is discarded (not dispatched to handlers)
4. If not duplicate: Message proceeds to filtering and handler dispatch

The duplicate check is performed per bus+channel+qos combination. Sequence numbers from different channels or QoS levels are tracked independently.

### Sequence Number Space

Sequence number spaces start at 1. When the receiving AEP Engine receives a message with sequence number 1, it treats this as a **sequence number space reset** and begins tracking from this new baseline. This prevents false duplicate detection when an upstream service restarts with a fresh sequence number space.

When an upstream service's state is wiped or reinitialized, it restarts its sending stream sequence at 1, which alerts downstream applications not to consider newly lowered sequence numbers as duplicates.

### Persistence Across Restarts

The AEP Engine persists the last sequence number received from each upstream sender as part of the microservice's HA state. This means:

* Duplicate detection operates **across service restarts**
* The receiving service remembers the last sequence number from each sender
* After restart, the service can continue detecting duplicates based on the persisted sequence number state

This persistence ensures that duplicate detection remains effective even after failover or restart scenarios.

{% hint style="info" %}
Duplicate detection is enabled by default. See [Configuring Duplicate Detection](/talon/developing-applications/configuring-the-runtime/message-flow/duplicate-detection) to learn how to disable it or customize its behavior.
{% endhint %}

## Monitoring

The AEP Engine tracks duplicate message statistics that can be monitored:

* **NumDupMsgsRcvd** - Total number of duplicate messages received and discarded

This metric is always 0 when `performDuplicateChecking=false`.

See [Engine Stats](/talon/operating-applications/monitoring/engine-statistics) for details on monitoring engine metrics.

## Prerequisites

For duplicate detection to work:

1. **Upstream sender must enable sequence numbers** - See [Message Sequencing](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages#message-sequencing) for how to configure sequence numbers on outbound messages
2. **Message bus binding must preserve sequence numbers** - The binding must transport sequence number metadata (enabled by default)

## See Also

* [Configuring Duplicate Detection](/talon/developing-applications/configuring-the-runtime/message-flow/duplicate-detection) - Configuration options
* [Message Sequencing](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages#message-sequencing) - How upstream services set sequence numbers
* [Filtering Messages](/talon/developing-applications/authoring-user-code/message-processing/filtering-messages) - Next step in the message processing pipeline
* [Engine Stats](/talon/operating-applications/monitoring/engine-statistics) - Monitoring duplicate detection metrics


# Filtering Messages

In situations where it is challenging or not possible to filter messages on the message bus itself using SMA's message routing and filtering features, a microservice can register a [message filter class](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageViewFilter.html) programmatically. The AEP engine triggers any registered message filter before dispatching messages to the microservice. Although programmatic message filtering is less efficient than bus-level filtering, it can significantly enhance performance by preventing filtered messages from entering the microservice engine's transaction stream and allowing them to be acknowledged upstream more quickly.

## Setting Message Filters <a href="#usingmessagefilters-usingmessagefilters" id="usingmessagefilters-usingmessagefilters"></a>

Use the `setMessageFilter()` method on an `AepEngine` to register a message filter. Message filters apply only to live messages while in the primary role. Filtered messages are acknowledged but not replicated nor logged to the transaction logs.

## Effects of a Message Filter <a href="#usingmessagefilters-effectsofamessagefilter" id="usingmessagefilters-effectsofamessagefilter"></a>

Filtered messages:

* Are acknowledged.
* Are not logged to the transaction log, inbound message log or replicated to cluster peers.
* Are not dispatched to application EventHandlers.
* Do not start new transactions or contribute to adaptive batching counts.
* Contribute to event received/processed and message received stats, but are accounted for in numMsgsFiltered. They do not undergo duplicate checking so filtered duplicates will not be accounted for duplicate counts.

## Message Filter Threading <a href="#usingmessagefilters-messagefilterthreading" id="usingmessagefilters-messagefilterthreading"></a>

The message filter is invoked from the same thread as application message handlers. Consequently they need not be coded in a thread safe fashion unless the application sets the same message filter instance on multiple AepEngines.

## Exceptions thrown from a Message Filter <a href="#usingmessagefilters-exceptionsthrownfromamessagefilter" id="usingmessagefilters-exceptionsthrownfromamessagefilter"></a>

If an exception is thrown by a MessageFilter, it is handled according to the [`AepEngine.AppExceptionHandlingPolicy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.AppExceptionHandlingPolicy.html) for this engine. If the policy is [`AepEngine.AppExceptionHandlingPolicy.LogExceptionAndContinue`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.AppExceptionHandlingPolicy.html#LogExceptionAndContinue), the message is not dispatched to application handlers and is acknowledged.

## HA Considerations <a href="#usingmessagefilters-haconsiderations" id="usingmessagefilters-haconsiderations"></a>

Any processing done in a message filter is done outside of a transaction and the effects of processing done in a message filter are not replicated to a backup. It is not legal for a message filter to modify the application's recoverable state or to use any of the engine's messaging facilities. Applications using Event Sourcing should take particular care in this regard as it could lead to divergence between a primary and its backup(s).


# Processing Messages

This section covers how to process messages in Talon microservices - the core of your application's business logic. All business logic in a Talon microservice is written in message handlers, which are methods that the AEP Engine invokes in response to inbound messages.

## Overview

Message processing in Talon follows a straightforward, event-driven model:

1. **Message Arrival**: An inbound message arrives on a channel that your microservice has subscribed to
2. **Duplicate Detection**: The AEP Engine checks sequence numbers to detect and discard duplicates
3. **Filtering**: Optional message filters determine if the message should be processed
4. **Handler Dispatch**: The AEP Engine dispatches the message to the appropriate handler(s) based on message type
5. **Business Logic Execution**: Your handler executes, reading the message, updating the microservice store, and creating outbound messages
6. **Transaction Commit**: When the handler returns, the AEP Engine commits the transaction, establishing consensus and sending outbound messages
7. **Message Acknowledgment**: The inbound message is acknowledged, completing the cycle

This simple model provides powerful guarantees: atomicity of store updates and message sends, exactly-once processing semantics, and automatic high availability through consensus with cluster members.

## The Message Processing Flow

Here's a visual representation of how messages flow through a Talon microservice:

```
Inbound Message → AEP Engine → Message Handler → Transaction Commit
                      ↓              ↓                    ↓
                  Dispatch       Business           Store Updates
                                  Logic            Outbound Messages
                                                    Consensus
```

### Key Characteristics

**Event-Driven Architecture**: Your code never polls or explicitly reads from queues. Instead, you write handlers that are invoked automatically when messages arrive.

**Single-Threaded Processing**: Each microservice processes messages on a single dispatch thread, eliminating concurrency issues and maximizing CPU cache efficiency. This is a fundamental design principle that enables Talon's performance.

**Automatic Transactions**: Every handler execution is automatically wrapped in a transaction. Store changes, outbound messages, and inbound message acknowledgment are atomic - they all succeed or all fail together.

**High Availability Built-In**: Depending on your chosen consensus model ([Event Sourcing](/talon/developing-applications/microservice-template/event-sourcing-template) or [State Replication](/talon/developing-applications/microservice-template/state-replication-template)), the AEP Engine automatically replicates data to backup instances and establishes consensus before committing transactions.

## What You'll Learn

This section covers the complete message processing lifecycle:

### [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages)

Learn how to write message handlers - the methods where all your business logic executes. Topics include:

* Writing your first handler
* Accessing and updating the microservice store
* Handler signatures and annotations
* Common patterns for message processing
* Programming fundamentals and rules you must follow
* Advanced topics like zero-garbage programming and transaction control

### [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages)

Learn how to create and send outbound messages from your handlers. Topics include:

* Creating and sending messages
* Message keys and topic resolution
* Channel configuration
* Unsolicited sends (messages not triggered by inbound messages)
* Send stability tracking

## Message Processing Models

Talon supports two consensus models that affect how message processing works:

**Event Sourcing** (recommended for most applications):

* Your microservice store consists of your own POJOs
* The AEP Engine replicates inbound messages to backup instances
* Backups replay messages to rebuild store state
* Provides the lowest latency and highest throughput
* Business logic must be deterministic

**State Replication**:

* Your microservice store uses ADM-generated classes
* The AEP Engine replicates store changes to backup instances
* Provides simpler programming model for certain use cases

For conceptual understanding of both models, see [Consensus Models](/talon/concepts-and-architecture/consensus-models).

## Best Practices

**Keep Handlers Fast**: Handlers should complete quickly (typically microseconds to milliseconds). Avoid blocking operations, long computations, or external I/O.

**Design for Horizontal Scaling**: Partition your data across multiple microservice instances using message keys. Each instance processes a subset of messages based on their keys.

**Embrace Determinism**: Especially with Event Sourcing, ensure your business logic produces the same results given the same inputs. Don't rely on system time, random numbers, or external state.

**Use Transactions Wisely**: For most cases, rely on automatic transaction management. Use [transaction controls](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions) only when you have specific requirements like incremental commits for large batches.

## Getting Started

If you're new to Talon message processing, we recommend reading in this order:

1. [Handling Messages - Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) - Essential rules you must understand
2. [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - How to write your first handler
3. [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - How to send outbound messages

## See Also

* [Consensus Models](/talon/concepts-and-architecture/consensus-models) - Understand Event Sourcing vs State Replication
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How consensus is established
* [Transactions](/talon/concepts-and-architecture/transactions) - Transaction semantics and guarantees
* [Configuring Messaging](https://github.com/neeveresearch/nvx-docs/blob/master/talon/developing-applications/authoring-user-code/configuring-messaging/README.md) - How to connect to message buses and subscribe to channels


# Handling Messages

This guide shows you how to write message handlers - the methods where all your business logic executes in a Talon microservice.

{% hint style="info" %}
**Note on Consensus Models**: This documentation focuses on **Event Sourcing**, which is Talon's primary consensus model. For **State Replication** examples and guidance, please refer to the [Rumi documentation](https://docs.rumi.systems). Rumi is the next major version (4.x) of the X Platform and provides more robust support for State Replication, including enhanced state modeling capabilities and improved developer tooling. To understand both models conceptually, see [Consensus Models](/talon/concepts-and-architecture/consensus-models).
{% endhint %}

{% hint style="warning" %}
**Required Reading**: Before writing message handlers, you must read [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals). This page covers essential rules about message immutability, single-threaded store access, and Event Sourcing determinism requirements that all handler code must follow.
{% endhint %}

## Overview

Message handlers are annotated methods that process inbound messages. When a message arrives, the AEP Engine dispatches it to the appropriate handler where your business logic executes. The handler reads data from the inbound message, updates the microservice store, and sends outbound messages.

## Writing a Message Handler

Here's a canonical message handler that demonstrates the key elements:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class OrderProcessor {

    // Injected by the platform
    private AepMessageSender messageSender;

    // Application-owned store (POJOs)
    private Map<String, Order> orders = new HashMap<>();

    @AppInjectionPoint
    public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    /**
     * Message handler for new order messages
     */
    @EventHandler
    public void onNewOrder(NewOrderMessage message) {
        // 1. Read data from inbound message
        String orderId = message.getOrderId();
        String symbol = message.getSymbol();
        int quantity = message.getQuantity();

        // 2. Read and update microservice store (POJOs)
        Order order = orders.get(orderId);
        if (order == null) {
            order = new Order();
            order.setOrderId(orderId);
            orders.put(orderId, order);
        }
        order.setSymbol(symbol);
        order.setQuantity(quantity);
        order.setStatus("PENDING");

        // 3. Create and send outbound message
        OrderAckMessage ack = OrderAckMessage.create();
        ack.setOrderId(orderId);
        ack.setStatus("ACCEPTED");
        ack.setTimestamp(System.currentTimeMillis());

        messageSender.sendMessage("order-acks", ack);

        // 4. Handler returns - transaction commits
        // The AEP Engine will:
        //   - Replicate inbound message to backup instances
        //   - Backup replays message to rebuild store
        //   - Establish consensus with cluster members
        //   - Commit the transaction
        //   - Send the outbound message
        //   - Acknowledge the inbound message
    }
}
```

## Handler Signature

A message handler must have this signature:

```java
@EventHandler
public void onMessageName(MessageType message) {
    // handler logic
}
```

**Key points:**

* Annotated with `@EventHandler`
* Must be `public`
* Return type must be `void`
* Takes exactly one parameter - the inbound message
* The message parameter type determines which messages this handler processes

### Handler Method Names

Method names are not significant - the handler is matched to messages by the parameter type. However, following a naming convention like `onMessageType` makes code more readable.

### Multiple Handlers for Same Message Type

You can have multiple handlers for the same message type:

```java
@EventHandler
public void validateOrder(NewOrderMessage message) {
    // Validation logic
}

@EventHandler
public void recordOrder(NewOrderMessage message) {
    // Recording logic
}
```

Both handlers will be invoked for each `NewOrderMessage`. The order of invocation is deterministic but should not be relied upon - handlers should be independent.

## Reading Inbound Messages

Access message fields using the generated getter methods:

```java
@EventHandler
public void onNewOrder(NewOrderMessage message) {
    String orderId = message.getOrderId();
    String symbol = message.getSymbol();
    int quantity = message.getQuantity();
    double price = message.getPrice();

    // Process the data...
}
```

**Important**: Inbound messages are read-only. See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) for rules about message immutability and lifecycle.

## Accessing Microservice Store

With Event Sourcing, your microservice store consists of POJOs that you manage directly. The store is private to your application and transparent to the Talon runtime:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class OrderProcessor {

    // Application-owned store (POJOs)
    private Map<String, Order> orders = new HashMap<>();
    private Map<String, Position> positions = new HashMap<>();

    @EventHandler
    public void onNewOrder(NewOrderMessage message) {
        // Access application store directly
        Order order = orders.get(message.getOrderId());

        if (order == null) {
            // Create new POJO
            order = new Order();
            order.setOrderId(message.getOrderId());
            orders.put(order.getOrderId(), order);
        }

        // Update data in store
        order.setQuantity(message.getQuantity());
        order.setStatus("PENDING");
    }
}
```

**Key points:**

* Store is your own POJOs, not ADM-generated
* Consensus established by replaying inbound messages on backup instances
* Store rebuilt on backup by replaying events
* Your business logic must be deterministic

**Store access rules:**

* Store can only be accessed from within a message handler (on the dispatch thread)
* Store changes must be deterministic - no reliance on external state like system time or random numbers
* See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals#single-threaded-state) for threading restrictions
* See [Event Sourcing Template](/talon/developing-applications/microservice-template/event-sourcing-template) for determinism requirements

## Sending Outbound Messages

Create and send messages using the `AepMessageSender`:

```java
@AppInjectionPoint
public void setMessageSender(AepMessageSender messageSender) {
    this.messageSender = messageSender;
}

@EventHandler
public void onNewOrder(NewOrderMessage message) {
    // Process order...
    Order order = orders.get(message.getOrderId());
    // ... update store ...

    // Create outbound message
    OrderAckMessage ack = OrderAckMessage.create();
    ack.setOrderId(message.getOrderId());
    ack.setStatus("ACCEPTED");

    // Send the message
    messageSender.sendMessage("order-acks", ack);
}
```

**When messages are sent:** Outbound messages are not immediately sent when you call `sendMessage()`. Instead:

1. The message is queued
2. The handler returns
3. The inbound message is replicated to backup instances
4. Backup instances replay the message to rebuild store
5. Consensus is established
6. The transaction commits
7. **Then** the outbound message is sent

This ensures that messages are only sent if the transaction succeeds, providing exactly-once semantics.

See [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) for details on message keys, channels, and unsolicited sends.

## Transaction Lifecycle

When a message handler executes, it runs within a transaction:

```java
@EventHandler
public void onNewOrder(NewOrderMessage message) {
    // Transaction starts (automatically)

    // 1. Read message data
    String orderId = message.getOrderId();

    // 2. Update store (POJO)
    Order order = new Order();
    order.setOrderId(orderId);
    orders.put(orderId, order);

    // 3. Queue outbound messages
    OrderAckMessage ack = OrderAckMessage.create();
    ack.setOrderId(orderId);
    messageSender.sendMessage("order-acks", ack);

    // Handler returns

    // 4. Transaction commits (automatically):
    //    - Inbound message replicated to cluster
    //    - Backup replays message to rebuild store
    //    - Consensus established
    //    - Outbound messages sent
    //    - Inbound message acknowledged
}
```

For details on how consensus works, see [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus).

For advanced transaction control, see [Controlling Transactions](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions).

## Common Patterns

### Pattern: Lookup or Create

```java
@EventHandler
public void onOrderUpdate(OrderUpdateMessage message) {
    String orderId = message.getOrderId();

    // Get existing or create new
    Order order = orders.get(orderId);
    if (order == null) {
        order = new Order();
        order.setOrderId(orderId);
        orders.put(orderId, order);
    }

    // Update
    order.setQuantity(message.getQuantity());
}
```

### Pattern: Conditional Send

```java
@EventHandler
public void onOrderUpdate(OrderUpdateMessage message) {
    Order order = orders.get(message.getOrderId());

    if (order != null) {
        order.setQuantity(message.getQuantity());

        // Send notification only if quantity exceeds threshold
        if (order.getQuantity() > 1000) {
            LargeOrderAlert alert = LargeOrderAlert.create();
            alert.setOrderId(order.getOrderId());
            alert.setQuantity(order.getQuantity());
            messageSender.sendMessage("alerts", alert);
        }
    }
}
```

### Pattern: Aggregate and Send

```java
@EventHandler
public void onTrade(TradeMessage message) {
    String symbol = message.getSymbol();

    // Update running totals in store
    DailyStats stats = dailyStats.get(symbol);
    if (stats == null) {
        stats = new DailyStats();
        stats.setSymbol(symbol);
        dailyStats.put(symbol, stats);
    }

    stats.setVolume(stats.getVolume() + message.getQuantity());
    stats.setTradeCount(stats.getTradeCount() + 1);

    // Send periodic snapshot
    if (stats.getTradeCount() % 100 == 0) {
        StatsSnapshot snapshot = StatsSnapshot.create();
        snapshot.setSymbol(symbol);
        snapshot.setVolume(stats.getVolume());
        snapshot.setTradeCount(stats.getTradeCount());
        messageSender.sendMessage("stats-snapshots", snapshot);
    }
}
```

### Pattern: Forwarding Messages

You cannot resend an inbound message directly. To forward a message, copy it first:

```java
@EventHandler
public void onOrder(OrderMessage message) {
    // Cannot do this:
    // messageSender.send("mirror", message); // ERROR!

    // Must copy first:
    OrderMessage copy = message.copy();
    messageSender.sendMessage("mirror", copy);
}
```

See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals#forwarding-messages) for details.

## Advanced Topics

### Using Savepoints

For long handlers or handlers that may fail partway through, you can use savepoints to commit work incrementally:

```java
@EventHandler
public void onBatch(BatchMessage message, MessageView view) {
    for (int i = 0; i < message.getItemCount(); i++) {
        // Process item
        processItem(message.getItem(i));

        // Savepoint every 100 items
        if (i % 100 == 0) {
            view.setSavePoint();
        }
    }
}
```

See [Using Savepoints](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions/using-savepoints) for details.

### Zero Garbage Programming

For ultra-low-latency applications, you can eliminate garbage collection pauses using zero-garbage techniques:

```java
@EventHandler
public void onOrder(OrderMessage message) {
    // Use XStrings instead of Strings
    XString symbol = message.getSymbol();  // No allocation

    // Use iterators instead of for-each
    XIterator<OrderLine> iter = message.iterateLines();
    while (iter.hasNext()) {
        OrderLine line = iter.next();  // No allocation
        // Process line...
    }
}
```

See [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) for details.

## See Also

* [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) - Core rules for message handlers
* [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - Creating and sending outbound messages
* [Controlling Transactions](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions) - Advanced transaction control
* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - Ultra-low-latency techniques
* [Event Sourcing Template](/talon/developing-applications/microservice-template/event-sourcing-template) - Event Sourcing model and requirements
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How consensus works (conceptual)


# Programming Fundamentals

## Overview

The Talon programming model is quite straightforward: *all* business logic in a Talon application is performed by event handlers that are invoked by Talon in response to received messages. The event handlers update state and generate outbound messages to interact with the outside world as a result of the business logic processing. Before diving into more details, this section describes the high level rules that developers must obey when working with Talon.

## Messaging

Talon's messaging APIs are discussed in more depth in sections on [Configuring Messaging](/talon/developing-applications/configuring-messaging) and [Authoring User Code](/talon/developing-applications/authoring-user-code) later in this manual. In the meantime keep the following rules in mind.

### Inbound Messages

An application may not modify inbound messages. Aside from being good practice, the platform relies on this immutability to do background journalling and replication of the message in parallel with your message handlers.

### Outbound Messages

Once an application sends an outbound message, it must not attempt to modify or reuse the message. This implies that any particular message instance can only be sent once. Applications may copy a message that has been sent and send it elsewhere. The reason for this is that the platform may concurrently serialize the message for replication or persistence purposes in parallel with your application logic.

### Messages Scoped to Handlers

In general, it is bad practice to hold onto a message or its embedded entity fields beyond the scope of a message handler. This is because the platform pools these messages and their underlying contents can be reset after return from a handler. This means that if applications intend to hold onto a message or a portion of the message in the microservice store, the safest approach is to copy them (or their contents) into the store.

In advanced use cases where an application must hold onto a message or one of its embedded entities for a short period of time, copy can be avoided by acquiring a reference to the message via a call to its `acquire()` method. If the application does acquire() the message it should later dispose() of the message to allow the platform to reuse it.

See [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) for more information.

### Forwarding Messages

Inbound messages may not be resent as outbound messages. Applications that need to forward an inbound message as an outbound message should first copy() the message and send the message copy:

```java
@EventHandler
public void onMyMessageReceived(MyMessage message) {
    MyMessage copy = message.copy();
    messageSender.send("mirror", copy);
}
```

## Threading

### Single Threaded State

Microservice models espouse a model where the store is private to each microservice. One of the primary motivators behind this model is that it avoids scalability bottlenecks that can arise from multiple processes contending to update the store. Talon microservices take this concept even further by making the microservice store private to a single processing thread. This restriction is key to Talon's ability to perform and scale by avoiding thread contention and best utilizing processors' CPU caches.

This means:

* Accessing the microservice's store outside of a message handler is forbidden. Any change to the microservice's store must be performed by the underlying dispatch thread in an event handler triggered by either [receipt](/talon/developing-applications/configuring-messaging/registering-message-interest) or [injection](/talon/developing-applications/authoring-user-code/message-injection) of a message.
* Applications should be designed to scale horizontally by partitioning state.

### No Blocking In Handlers

Talon microservices are optimized for stream-oriented message processing. This means that message handlers should never perform blocking operations. Additionally, since Talon applications have a single-threaded state model, Talon is not optimized for long running, compute intensive operations.

In practice this is generally not an issue for the class of applications targeted by Talon.

* Talon handles I/O operations for applications, so applications should not be working with blocking I/O resources.
* If an application needs to make a blocking callout to interact with a remote resource, that interaction should be done asynchronously via messaging.
* For compute intensive tasks triggered by processing a message, applications may schedule the work on another thread and [inject the results](/talon/developing-applications/authoring-user-code/message-injection) back into the engine to update the microservice store.

## State Tree Limitations

There are several restrictions enforced by Talon's state model that application developers must be aware of when designing state models and authoring applications. Understanding these limitations is critical for building correct applications.

### Single Parent Restriction

An entity may only be placed in the state tree as the field of a **single** parent object. Using the same object instance in the tree in multiple locations is not supported.

```java
ParentObject p1 = EntityFactory.createParentObject();
ParentObject p2 = EntityFactory.createParentObject();

ChildObject c1 = EntityFactory.createChildObject();
p1.setChild(c1);
p2.setChild(c1); // Not supported - throws IllegalStateException
assertTrue(p1.getChild() == p2.getChild()); // Will fail
```

{% hint style="warning" %}
Attempting to set an entity as a field of another entity when it is already set as a field elsewhere in the state tree will result in an `IllegalStateException` at runtime.
{% endhint %}

**Workaround**: Store entities in a Map and reference them by ID. For example, if a Customer object needs to be referenced in multiple places in the state tree, hold all Customers in a Map and reference each Customer by its ID from multiple locations.

### Multiple Entity Fields of Same Type

Transactional entities don't currently support multiple fields of the same entity type. The following model is **not supported** due to limitations in the underlying transaction machinery:

```xml
<model defaultFactoryId="1" namespace="com.example.models.state">
  ...
  <entities>
    <entity name="ParentObject" id="1">
      <field name="childField1" type="ChildObject" id="1"/>
      <field name="childField2" type="ChildObject" id="2"/>
    </entity>
    <entity name="ChildObject" id="2"/>
  </entities>
</model>
```

This will yield an error during code generation:

```
Duplicate entity fields of the same type 'com.example.models.state.ChildObject'...
an entity cannot have two non-embedded entity fields of the same type
```

{% hint style="info" %}
**Exception**: It **is** permissible to declare multiple **embedded** entity fields of the same type.
{% endhint %}

### State Tree Cycles

As a corollary to the Single Parent Restriction, cycles in the state tree are not supported (including self-references):

```java
ParentObject parent = EntityFactory.createParentObject();

ChildObject child = EntityFactory.createChildObject();
parent.setChild(child);
child.setParent(parent); // Not supported - creates cycle
```

{% hint style="info" %}
The Application Data Modeler will, by default, not permit data models that contain cycles in the resultant state tree.
{% endhint %}

### Primitive Collections

It is not currently possible to define collections with primitive value types. Use boxed types (e.g., `Integer` instead of `int`) for collection elements.

### Inheritance and Polymorphism

Inheritance is not currently supported in Talon's state model. However, the platform supports [entity inlining](/talon/developing-applications/modeling-messages-and-state/the-modeling-language) to provide a form of polymorphism:

```xml
<entity name="ObjectA" factoryid="1" id="1">
  <field name="field1" type="Integer"/>
</entity>

<entity name="ObjectB" factoryid="1" id="2" inline="ObjectA">
  <field name="field2" type="Integer"/>
</entity>
```

Application developers should be aware that polymorphism can have a significant impact on Java performance.

## Event Sourcing Considerations

Of Talon's two High Availability models ([State Replication](/talon/developing-applications/microservice-template/state-replication-template) and [Event Sourcing](/talon/developing-applications/microservice-template/event-sourcing-template)), the more advanced is Event Sourcing. The Event Sourcing model is best suited to applications that require very low latency, but it does impose some additional programming restrictions.

An application using Event Sourcing must be able to exactly reconstruct the same state and outbound messaging stream by replaying the same sequence of inbound events. This implies that business logic that updates state *must not* rely on any environmental state when making updates to its recoverable state or outbound messages. For example, setting the system time in an outbound message would not produce the same outbound message when the event stream is replayed at a later date or on a different system.

See [Event Sourcing Template](/talon/developing-applications/microservice-template/event-sourcing-template) for more information on working with Event Sourcing and considerations for environment replication.


# Coding for Zero Garbage

## Overview

Talon's pooling, messaging and state types are designed to support zero garbage operation once the system reaches steady state. Relying on JVM garbage collection can introduce unpredictable pauses, particularly when applications have large heap sizes. Zero garbage programming eliminates allocation that triggers GC pauses enabling applications to operate at consistently low latencies.

This section discusses techniques and best practices for achieving zero garbage operation with Talon microservices.

## Weighing Alternatives

Achieving zero garbage operation is an advanced technique. The cost of engineering software for zero garbage operation can have trade offs in the complexity of the code as the user must be conscious about re-using application objects. In some cases tuning the JVM or using a GC-optimized JVM may be a more cost effective approach. For example, using Azul's Zing JVM provides near pauseless GC with significantly lower development complexity than zero-garbage techniques.

Before pursuing zero garbage techniques, consider:

* **Performance Requirements**: Is sub-millisecond latency critical? Standard GC tuning may suffice for many use cases
* **Development Complexity**: Zero garbage programming requires careful lifecycle management and reference counting
* **Maintenance Burden**: More complex code is harder to maintain and evolve
* **JVM Alternatives**: Modern low-latency JVMs like Zing may provide sufficient GC pause reduction

## Zero Garbage Techniques

Talon provides several facilities for achieving zero garbage operation:

* [XStrings](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/xstrings) - Zero garbage string handling with pooled, mutable strings
* [Embedded Entities](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/embedded-entities) - Lifecycle-managed objects for complex message composition
* [Array Accessors](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/array-accessors) - XIterator pattern for allocation-free array iteration
* [Application Object Pooling](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/application-object-pooling) - Creating custom pooled objects with reference counting
* [Tuning Pools](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/tuning-pools) - Configuration and optimization of pool parameters

## See Also

* [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals) - Core programming rules including messaging and threading constraints
* [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Message handler best practices
* [Configuring Adaptive Batching](/talon/developing-applications/configuring-the-runtime/transactions/adaptive-batching) - Transaction batching for throughput optimization


# XStrings

XStrings provide a zero garbage alternative to Java Strings for ultra-low-latency applications. XStrings are mutable, string-like objects that can be pooled and preallocated so that string handling neither allocates nor promotes during message processing.

## Overview

An `XString` is a mutable, zero garbage alternative to `java.lang.String`. Java Strings are immutable and allocate on every manipulation, which makes them unsuitable for values that change on each message. An `XString` can be written in place, compared without allocating, and copied to and from messages without creating garbage.

Eliminating garbage is only half the problem. Objects held in application state survive long enough to be promoted across heap generations, and promotion is expensive. Avoiding it requires that the strings held in state be pooled and preallocated rather than created as needed. That is what a *poolable* string type provides.

## Declaring a Poolable String

Poolable string types are declared in the ADM model, not in application code. A String field or semantic type carrying `poolable="true"` causes the code generator to emit a dedicated subclass of `XString` for it, together with the factory used to pool and preallocate instances:

```xml
<types>
    <type name="ComplianceId" base="String" length="24" poolable="true"
          doc="Identifier attached to an order for compliance reporting."/>
</types>

<fields>
    <field name="complianceId" type="ComplianceId" id="1"/>
</fields>
```

A field may also be marked poolable directly, in which case the generated type is named after the field rather than the semantic type. The declaration syntax, the generated type, and the naming rules are covered in full under [Poolable String Types](/talon/developing-applications/modeling-messages-and-state/the-modeling-language#poolable-string-types).

## Working with Poolable Strings

Fields of a poolable string type are given accessors that copy the value in and out rather than handing out a reference:

| Method                        | Description                                                                                                                                                        |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `getXXXTo(factory)`           | Copies the field's value into an instance sourced from the given factory and returns it. Zero garbage when the factory is pool backed and has instances available. |
| `setXXXFrom(value)`           | Sets the field from an already encoded `XString`, avoiding a re-encode.                                                                                            |
| `getXXX()` / `setXXX(String)` | The plain String accessors, which allocate. Useful outside the hot path.                                                                                           |

There are two patterns in common use.

### Holding a value in application state

Source the string from a preallocated factory and copy the message value into it. This is the pattern to use whenever the value outlives the handler:

```java
private final ComplianceId.Factory complianceIdFactory =
    ComplianceId.newFactory("complianceId", 24, true, 1024, false, false);

@EventHandler
public void onNewOrder(NewOrder order, OrderState state) {
    // Copy the value out of the message into an instance from the pool.
    ComplianceId complianceId = order.getComplianceIdTo(complianceIdFactory);

    // Copy it into an outbound message.
    OrderAccepted accepted = OrderAccepted.create();
    accepted.setComplianceIdFrom(complianceId);

    // Retain it in application state.
    state.setComplianceId(complianceId);
}
```

Copying out of the message is not optional here. If the message itself is pooled, its field values are reset once the handler returns, so a reference retained in state would be left holding a value that no longer belongs to it.

### Using a working variable

Where the value does not outlive the handler, a single reusable instance is enough:

```java
private final ComplianceId complianceIdTemp = ComplianceId.create(true);

@EventHandler
public void onNewOrder(NewOrder order) {
    // Safe to overwrite on each invocation because handlers are single threaded
    // and the value is not retained beyond this call.
    order.getComplianceIdTo(complianceIdTemp);

    OrderAccepted accepted = OrderAccepted.create();
    accepted.setComplianceIdFrom(complianceIdTemp);
}
```

Note that this pattern retains the reference and so does not exercise the pooling capability of the type. It avoids allocation, not promotion.

## Preallocation

Preallocation is a property of the *factory*, not of the field. A factory created with a preallocation count fills its pool up front, so that instances taken during message processing are already resident and do not have to be allocated and later promoted:

```java
// name, stringLength, pooled, preallocationCount, threaded, isNative
ComplianceId.Factory factory =
    ComplianceId.newFactory("complianceId", 24, true, 1024, false, false);
```

* `stringLength` sizes the backing buffer of each instance. Size it for the longest value you expect. A value that exceeds it forces the backing storage to grow, which allocates.
* `pooled` determines whether the factory is backed by a pool at all. Preallocation is only meaningful when it is.
* `preallocationCount` is the number of instances created up front.
* `threaded` makes the backing pool thread safe. Leave it false for the single threaded handler path.
* `isNative` selects native backing buffers where enabled and supported.

Each generated type also exposes a default `FACTORY` for cases where a configured factory is not warranted.

Pools created this way are configurable at runtime alongside the platform's other pools. See [Tuning Pools](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/tuning-pools).

## A Note on RawString

`XString` supersedes `RawString`, which remains present but is deprecated. New code should use `XString` and the generated poolable types; existing uses of `RawString` should migrate.

## See Also

* [Poolable String Types](/talon/developing-applications/modeling-messages-and-state/the-modeling-language#poolable-string-types) - declaring poolable strings and the code the generator emits
* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - overview of zero garbage techniques
* [Tuning Pools](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/tuning-pools) - configuring the pools that back preallocated types
* [Embedded Entities](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/embedded-entities) - lifecycle-managed objects for complex messages


# Embedded Entities

Embedded entities are lifecycle-managed objects that enable zero-garbage composition of complex messages. They are reusable, pooled objects embedded within messages or state objects.

## Overview

Embedded entities allow you to compose complex, nested data structures within messages and state while maintaining zero-garbage characteristics. Instead of allocating new objects for each nested structure, embedded entities are pooled and reused across message processing cycles.

## Declaring Embedded Entities

Embedded entities are declared in ADM using the `entity` type with the `embedded="true"` attribute:

```xml
<entity name="OrderLine" embedded="true">
  <string name="symbol"/>
  <int name="quantity"/>
  <double name="price"/>
</entity>

<message name="Order">
  <string name="orderId"/>
  <OrderLine name="line"/>
</message>
```

## Lifecycle Management

Embedded entities have a strict lifecycle that must be managed correctly to achieve zero-garbage operation. The ADM code generator produces methods to manage this lifecycle:

| Method           | Description                                                                                                                        | Returns             |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------- |
| `getXXX()`       | Returns a reference to the embedded entity. Object remains owned by the parent.                                                    | Read-only reference |
| `takeXXX()`      | Removes the embedded entity from the parent and transfers ownership to the caller. The parent's field becomes null.                | Transferred entity  |
| `lendXXX()`      | Returns a reference for modification. Object ownership remains with the parent.                                                    | Mutable reference   |
| `setXXX(Entity)` | Sets the embedded entity. If a non-null entity already exists, it is disposed. The provided entity is copied if not already owned. | void                |

## Using getXXX()

The `getXXX()` accessor returns a read-only reference to the embedded entity. The parent retains ownership and the returned object should not be modified:

```java
@EventHandler
public void onOrder(Order order) {
    OrderLine line = order.getLine();
    if (line != null) {
        String symbol = line.getSymbol();
        // Read fields, but don't modify
    }
}
```

**Important**: The returned entity may be reset or reused after the message handler returns. Do not hold references to embedded entities beyond the scope of the handler.

## Using takeXXX()

The `takeXXX()` method transfers ownership of the embedded entity from the parent to the caller:

```java
@EventHandler
public void onOrder(Order order) {
    // Take ownership of the line
    OrderLine line = order.takeLine();

    // Parent's line field is now null
    assert order.getLine() == null;

    // We now own the line and can modify it
    line.setQuantity(100);

    // Must dispose when done to return to pool
    line.dispose();
}
```

After calling `takeXXX()`, the parent's field becomes null, and the caller is responsible for disposing the entity when finished.

## Using lendXXX()

The `lendXXX()` method returns a reference for modification while the parent retains ownership:

```java
@EventHandler
public void onOrder(Order order) {
    // Get a mutable reference
    OrderLine line = order.lendLine();
    if (line == null) {
        // Create if doesn't exist - parent creates and owns
        line = order.setLine(new OrderLineXbufEntity());
    }

    // Modify directly - parent still owns
    line.setQuantity(100);
    line.setPrice(150.25);

    // No dispose needed - parent still owns
}
```

The `lendXXX()` pattern is the most common for zero-garbage updates to embedded entities.

## Using setXXX()

The `setXXX()` method sets the embedded entity value:

```java
@EventHandler
public void onOrder(Order order) {
    // Create a new line
    OrderLine line = new OrderLineXbufEntity();
    line.setSymbol("AAPL");
    line.setQuantity(100);

    // Set on parent - parent takes ownership
    order.setLine(line);

    // Parent now owns it, no dispose needed
}
```

If an entity already exists in the parent when `setXXX()` is called, the old entity is automatically disposed.

## Serialization and Deserialization

Embedded entities are automatically serialized and deserialized with their parent message. When a message containing embedded entities is sent, the entities are serialized as part of the message payload. When received, they are deserialized and populated from the message content.

## Pass-Through Fields

Embedded entities support "pass-through" behavior where incoming message entities can be efficiently transferred to outbound messages without deep copying:

```java
@EventHandler
public void onOrder(Order incomingOrder, MessageView view) {
    // Create outbound order
    OrderAck ack = new OrderAckXbufMessage();

    // Take from incoming (transfers ownership)
    OrderLine line = incomingOrder.takeLine();

    // Set on outgoing (transfers ownership again)
    ack.setLine(line);

    // Send ack
    view.send(ack);
}
```

This pattern avoids copying the embedded entity's data, achieving zero-garbage pass-through.

## Common Patterns

**Creating if Null**:

```java
OrderLine line = order.lendLine();
if (line == null) {
    order.setLine(new OrderLineXbufEntity());
    line = order.lendLine();
}
line.setQuantity(100);
```

**Clearing an Embedded Entity**:

```java
OrderLine line = order.takeLine();
if (line != null) {
    line.dispose(); // Returns to pool
}
// order.getLine() is now null
```

**Updating Multiple Fields**:

```java
OrderLine line = order.lendLine();
if (line != null) {
    line.setSymbol("AAPL");
    line.setQuantity(100);
    line.setPrice(150.25);
}
```

## See Also

* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - Overview of zero garbage techniques
* [XStrings](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/xstrings) - Zero garbage string handling
* [Array Accessors](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/array-accessors) - Allocation-free array iteration


# Array Accessors

Zero garbage array accessors enable iteration over message array fields without allocating iterator objects. They use the XIterator pattern to provide zero-garbage traversal of arrays.

## Overview

When a message or state object contains an array field, traditional Java iteration using `for-each` or `Iterator` creates temporary objects that contribute to garbage. Talon's zero garbage array accessors provide a way to iterate over arrays without any allocations.

## XIterator Pattern

The XIterator pattern provides zero-garbage iteration through reusable iterator objects. When ADM generates code for array fields, it produces `iterateXXX()` methods that return an XIterator.

## Generated Methods

For an array field declared as:

```xml
<message name="Order">
  <OrderLine name="lines" array="true"/>
</message>
```

The ADM code generator produces:

| Method                   | Description                                     |
| ------------------------ | ----------------------------------------------- |
| `getLines()`             | Returns a List view of the array (allocates)    |
| `getLines(int index)`    | Returns the element at the specified index      |
| `iterateLines()`         | Returns an XIterator for zero-garbage iteration |
| `addLine(OrderLine)`     | Adds an element to the array                    |
| `removeLines(int index)` | Removes element at index                        |
| `clearLines()`           | Removes all elements                            |

## Zero Garbage Iteration

Use the XIterator for zero-garbage array traversal:

```java
@EventHandler
public void onOrder(Order order) {
    // Zero garbage iteration
    XIterator<OrderLine> iter = order.iterateLines();
    while (iter.hasNext()) {
        OrderLine line = iter.next();
        // Process line
        double value = line.getQuantity() * line.getPrice();
    }
}
```

The XIterator is reused across calls and does not allocate.

## Use Cases

**Filtering**:

```java
XIterator<OrderLine> iter = order.iterateLines();
while (iter.hasNext()) {
    OrderLine line = iter.next();
    if (line.getQuantity() > 100) {
        // Process large orders
    }
}
```

**Aggregation**:

```java
double total = 0.0;
XIterator<OrderLine> iter = order.iterateLines();
while (iter.hasNext()) {
    OrderLine line = iter.next();
    total += line.getQuantity() * line.getPrice();
}
```

**Modifying Elements**:

```java
XIterator<OrderLine> iter = order.iterateLines();
while (iter.hasNext()) {
    OrderLine line = iter.next();
    // Modify in place
    line.setPrice(line.getPrice() * 1.1); // Apply 10% markup
}
```

## Comparison with Standard Java Iteration

**Standard (allocates)**:

```java
// Creates iterator object
for (OrderLine line : order.getLines()) {
    processLine(line);
}
```

**Zero Garbage**:

```java
// Reuses iterator object
XIterator<OrderLine> iter = order.iterateLines();
while (iter.hasNext()) {
    OrderLine line = iter.next();
    processLine(line);
}
```

The zero-garbage approach reuses the same XIterator instance on each call to `iterateXXX()`, eliminating allocations.

## See Also

* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - Overview of zero garbage techniques
* [Embedded Entities](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/embedded-entities) - Lifecycle-managed objects for complex messages
* [Application Object Pooling](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/application-object-pooling) - Creating custom pooled objects


# Application Object Pooling

Beyond platform-provided pooling for messages and embedded entities, applications can use Talon's pooling facilities for custom objects to achieve zero-garbage operation.

## Overview

Platform generated messages and entities generated with Xbuf encoding support pooling out of the box. Latency sensitive applications can also use the platform's pooling facilities for user objects. Using the platform's pooling facilities has several advantages:

* Pool usage is tracked and reported by XVM stats and heartbeats providing visibility into leaks
* Platform pools support preallocation out of the box

{% hint style="info" %}
Using the platform's pooling facilities is considered an advanced feature of Talon. It is only recommended for users that have the most stringent latency and throughput requirements. A viable alternative to pooling may also be to use the low latency distribution of Talon which bundles Azul's Zing JVM which provides near pauseless GC. This is a lower complexity approach than pooling.
{% endhint %}

## Coding Pooled Objects

Making an object poolable involves the following steps:

1. Implement `com.neeve.util.UtlPool.Item`
2. Create a `UtlPool.Factory` for creating new Items or arrays of Items (used by the pool to construct new instances)

## Sample Object

The following example shows how to make an Order object poolable:

```java
final public class Order implements Item<Order> {

    final private static class OrderFactory implements UtlPool.Factory<Order> {
        @Override
        final public Order createItem(final Object context) {
            return new Order();
        }

        @Override
        final public Order[] createItemArray(final int size) {
            return new Order[size];
        }
    }

    // TODO: your member variables
    private int orderQuantity;

    // order pool - member variable to store pool this Order belongs to
    private UtlPool<Order> pool;

    /**
     * Creates a new order pool with the provided number of preallocated orders.
     *
     * @param orderPreallocateCount The number of orders to preallocate in the pool
     * @param poolName The name of the pool
     * @return A new order pool
     */
    public static UtlPool<Order> createPool(int orderPreallocateCount, String poolName) {
        final UtlPool<Order> orderPool = UtlPool.create(
            "order",
            poolName,
            new OrderFactory(),
            UtlPool.Params.create()
                .setThreaded(false)
                .setInitialCapacity(orderPreallocateCount)
                .setPreallocate(true)
        );
        return orderPool;
    }

    private Order() {
        // initialization
        init();
    }

    /**
     * Implementation of {@link Item#init()}
     *
     * This method cleans a pool item when it is recycled to the pool or
     * added for the first time.
     */
    @Override
    final public Order init() {
        // TODO: reset your variables
        orderQuantity = -1;
        return this;
    }

    /**
     * Implementation of {@link Item#setPool}
     *
     * Called by the pool to mark that this instance belongs to it.
     */
    @Override
    final public Order setPool(UtlPool<Order> pool) {
        this.pool = pool;
        return this;
    }

    /**
     * Implementation of {@link Item#getPool}
     *
     * Gets the pool that this instance belongs to.
     */
    @Override
    final public UtlPool<Order> getPool() {
        return pool;
    }
}
```

Note the following:

* The `UtlPool.Factory` is implemented as a private inner class (though this is not mandatory)
* The Order object is a factory for its own pool via the static `createPool()` method (also not mandatory)
* The pool is created as non-threaded via `UtlPool.Params` passed in. This means only a single thread can take and/or put items into the returned pool. Because user code is single threaded, it is often acceptable to create pools as single threaded, particularly for preallocation use cases
* The Factory is created with 2 String parameters: the pool type and the pool name. The combination must be unique within the JVM

## Implementing Reference Counting

The `UtlPool.Item` interface does not impose reference counting semantics, but you can add such behavior:

```java
final public class Order implements Item<Order> {

    // SNIP - see the pooling code above

    // reference count for this object
    final protected AtomicInteger ownershipCount = new AtomicInteger(1);

    // order pool - member variable to store pool this Order belongs to
    private UtlPool<Order> pool;

    /**
     * Acquires a reference to the object.
     */
    @Override
    final public void acquire() {
        final int val = ownershipCount.incrementAndGet();
        if (val <= 1) {
            throw new IllegalStateException("attempt to acquire an already disposed Order!");
        }
    }

    /**
     * Returns the current ownership count of this object.
     */
    @Override
    final public int getOwnershipCount() {
        return ownershipCount.get();
    }

    /**
     * Implementation of {@link Item#init()}
     *
     * This method cleans a pool item.
     */
    @Override
    final public Order init() {
        // restore ownership count back to 1
        ownershipCount.getAndSet(1);
        // TODO: reset any other field values here
        return this;
    }

    /**
     * Disposes this order object. When its count drops to
     * 0 it will be returned to its pool (if pooled).
     */
    @Override
    public int dispose() {
        final int val = ownershipCount.decrementAndGet();
        if (val < 0) {
            throw new IllegalStateException("attempt to dispose an already disposed Order!");
        }

        if (val == 0 && pool != null) {
            pool.put(this);
        }
        return val;
    }
}
```

## Using Pools

Once you have created your pooled object, you can create pools and use the objects:

```java
UtlPool<Order> orderPool = Order.createPool(100, "order-pool");

Order order = orderPool.get(null); // ownershipCount is 1
// Do something
order.dispose(); // ownershipCount --> 0, object returned to pool
```

If the object will be passed off to another thread where it may be worked on in parallel, then you should `acquire()` a reference before transferring ownership:

```java
Order order = orderPool.get(null); // ownershipCount is 1

// Hand off to a parallel thread
order.acquire(); // ownershipCount is 2
executor.execute(() -> {
    // Do stuff
    order.dispose();
});

// Do more things

order.dispose(); // ownershipCount --> 0? object returned to pool
```

## Configuring Pools At Runtime

{% hint style="warning" %}
Note that config driven pool configuration is still in incubation and the following is subject to change.
{% endhint %}

In the pooled Order example above, the pool was hardcoded to preallocate a specific number of entries. It is possible to override programmatic configuration using environment variables. This can be achieved by setting properties of the form `nv.pool.<poolKey>.<propertyName>` where:

**poolKey** is: The `<poolType>.<poolName>` (the same as reported in pool stats without the trailing `.instanceId` suffix). In the example above this would be "order.order-pool"

**propertyName** is one of the bean properties on [`UtlPool.Params`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/util/UtlPool.Params.html):

* `initialCapacity`
* `maxCapacity`
* `threaded`
* `preallocate`
* `detachedWash`

Pool properties can be configured in DDL as:

```xml
<env>
  <!-- pool parameters -->
  <nv>
    <pool>
      <!-- enable overrides of programmatically set values -->
      <overrideparamsfromenv>true</overrideparamsfromenv>

      <!-- configure pool of type "order" and name "order-pool" -->
      <order.order-pool>
        <initialCapacity>10000</initialCapacity>
        <preallocate>true</preallocate>
      </order.order-pool>
    </pool>
  </nv>
</env>
```

## Pooling Configuration Properties

The following table summarizes pooling properties that can be set in env. Check the javadoc for [UtlConstants](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/util/UtlConstants.html) for the most up to date values along with additional advanced properties.

| Property Name                 | Default | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| ----------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| nv.pool.shouldpool            | true    | Property that controls whether pooling is enabled globally. Globally disabling pools is not usually recommended as it can have adverse consequences on memory management. This property is mainly provided for troubleshooting purposes.                                                                                                                                                                                                                                                                                                                                                          |
| nv.pool.sourceparamsfromenv   | true    | Property that controls whether pool parameters can be sourced from the environment. When `true`, calls to create pools will result in `Params.load(poolkey, UtlEnv.getProps(), false)` being invoked to apply pool parameters that haven't already been set explicitly. To override programmatically set values see `nv.pool.overrideparamsfromenv`. This setting is currently classified as experimental and is subject to change.                                                                                                                                                               |
| nv.pool.overrideparamsfromenv | false   | Property that controls whether pool parameters can be overridden from the environment. By default, when pool parameters are sourced from the environment, they will not override values explicitly set programmatically. Setting this property forces the environment specified values to take precedence. This property should be used with extreme care as overriding programmatically set pool parameters can have adverse effects. This property takes no effect if `nv.pool.sourceparamsfromenv` is disabled. This setting is currently classified as experimental and is subject to change. |

## See Also

* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - Overview of zero garbage techniques
* [Tuning Pools](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/tuning-pools) - Pool configuration and optimization
* [Embedded Entities](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/embedded-entities) - Platform-provided pooled entities


# Tuning Pools

To achieve zero garbage operation in steady state, Talon pools objects to avoid allocations. This section describes how to tune pools for optimal performance.

## Overview

Pools created by the platform start out with no objects preallocated, so it is important for performance sensitive applications to drive warm-up traffic at startup not only to allow for JIT optimization to kick in, but also to make sure pools are grown to reach an equilibrium state where enough instances have been allocated to satisfy the number of objects in use at a given time.

To reach such an equilibrium state, applications should push traffic at rates higher than expected volume to ensure that pool sizes grow to a level that can accommodate spikes in traffic. In cases where application warm-up isn't sufficient for reaching pool equilibrium, pools can be manually configured.

Some cases where it is desirable to manually configure pool parameters include:

* When performance (throughput and latency) is less important than memory footprint, disabling pools can avoid memory overhead where objects might otherwise sit in a pool unused
* Cases where it is challenging to drive warm-up traffic to the point where pools reach optimal capacity - manual configuration with preallocation is desirable
* Pools used purely in a preallocated fashion (e.g., applications that expect a given number of Order objects per day may want to start with those objects preallocated)
* In some cases initial bursts of traffic at application startup cause pools of seldom used types to grow to a large size but then remain dormant taking up wasted memory - it would be desirable to limit the pool's capacity

## General Pooling Configuration

Pooling of platform internal objects and embedded entities generated with Xbuf encoding is enabled by default. ADM Messages and State Entities can also be pooled when generated with Xbuf encoding, but pooling of these types is currently only enabled with `nv.optimizefor=throughput` or `nv.optimizefor=latency`. In certain cases it may be desirable to change this behavior at a more granular level.

| Parameter          | Default                                              | Comments                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| ------------------ | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| nv.pool.shouldpool | true                                                 | This property can be set to false to globally disable all pooling including platform internal pools. Setting this value to false can adversely affect throughput and latency but can be useful in lowering memory overhead for lower performance use cases in which garbage collection costs are low.                                                                                                                                                                                                                                                                                   |
| nv.pkt.shouldpool  | false (true when nv.optimizefor=throughput\|latency) | This property globally enables or disables packet pooling. ADM generated Xbuf Messages or Entities are backed by and pooled with their backing packet objects, consequently this parameter controls pooling for these types. Note that *embedded* entities are not backed by a packet and are pooled independently of the Message or Entity in which they are contained, so this property does not affect embedded entities. Packet pooling impacts several areas of the platform including transaction logs and cluster replication as both operate using packets to frame their data. |

## Configuring Specific Pools

A pool is uniquely named in a JVM as `<pooltype>.<poolname>.<poolinstanceid>`. For example, a native 256 byte platform IOBuffer named "iobuf.native-256.23" has a pooltype of "iobuf", a pool name of "native-256" and is suffixed with a JVM unique instance id.

The following output from enabling pool stats trace shows some example pool names:

```
[Pool Stats]
PUT   DPUT  GET   DGET  HIT   DHIT  MISS  DMISS GROW  DGROW EVIC  DEVIC DWSH  DDWSH SIZE  PRE   CAP   NAME
612K  937   11.3M 947   612K  936   10.7M 11    0     0     0     0     0     0     1     0     1024  iobuf.native-256.23
8302  40    50670 41    8302  40    42368 1     0     0     0     0     0     0     0     0     1024  iobuf.native-512.24
62    20    1.5M  0     31    0     1.5M  0     0     0     0     0     0     0     31    0     1024  packet.MyMessageXbufPacket.71.1.266
62    20    1.5M  0     31    0     1.5M  0     0     0     0     0     0     0     31    0     1024  xbuf.entity.MyEntityXbufEntity.301.199.267
```

Pools are configured by specifying `nv.pool.<poolidentifier>.<propertyname>` where the identifier can be either the `pooltype` or the `pooltype.poolname`.

## Pool Config Identifiers

Example pool identifiers from the above stats output:

* "iobuf" - applies to all IOBuffer pools
* "iobuf.native-256" - applies only to native 256 byte IOBuffer pools
* "packet" - applies to all packet pools
* "packet.MyMessageXbufPacket.71.1" - applies to the packet type backing the Xbuf generated MyMessage class with a factory id of 71 and type id of 1 (the pool instance id of 266 has no bearing on configuration and will change from run to run)
* "xbuf.entity" - applies to all Xbuf embedded entities
* "xbuf.entity.MyEntityXbufEntity.301.199" - applies to the embedded Xbuf MyEntity class with factory id 301 and type id 199

When both a type level and type+poolname property is configured, the higher granularity 'poolname' property takes precedence.

## Pool Configuration Properties

The following environment variables can be used to configure a specific pool type using the above identifier:

| Parameter                                 | Default            | Comments                                                                                                                                                                   |
| ----------------------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| nv.pool.\<poolIdentifier>.initialCapacity | 1024               | The initial size of the array to hold pool items that are not in use.                                                                                                      |
| nv.pool.\<poolIdentifier>.maxCapacity     | Integer.MAX\_VALUE | The maximum number of slots for items not in use. If an item is returned to a pool and there are no slots left, it is evicted and becomes eligible for garbage collection. |
| nv.pool.\<poolIdentifier>.threaded        | true               | Whether or not the pool will be safe for access by multiple threads.                                                                                                       |
| nv.pool.\<poolIdentifier>.preallocate     | false              | When true, pool is filled to its initial capacity with newly created items when the pool is created. When false, items are created on demand.                              |
| nv.pool.\<poolIdentifier>.detachedWash    | false              | When items are returned to a pool, their fields are reset. When true, this makes items returned to the pool eligible for cleanup on a detached thread.                     |

{% hint style="warning" %}
**Note on above defaults**: Each pool is created programmatically and may set its own default values; the above defaults apply to pools that haven't altered the default value.
{% endhint %}

## DDL Pool Configuration Examples

The following configuration shows an example of configuring preallocation for the MyEntity Xbuf entity type:

```xml
<env>
  <!-- pool parameters -->
  <nv>
    <pool>
      <xbuf.entity.MyEntityXbufEntity.301>
        <initialCapacity>16384</initialCapacity>
        <preallocate>true</preallocate>
      </xbuf.entity.MyEntityXbufEntity.301>
    </pool>
  </nv>
</env>
```

## See Also

* [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) - Overview of zero garbage techniques
* [Application Object Pooling](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage/application-object-pooling) - Creating custom pooled objects
* [Configuring the Microservice Runtime](/talon/developing-applications/configuring-the-runtime) - Runtime configuration options


# Controlling Transactions

While Talon handles transactions automatically for most use cases, it also provides programmatic controls that allow you to fine-tune transaction behavior from within your message handlers. These capabilities are useful for advanced scenarios where you need more control over transaction boundaries and commit behavior.

## Overview

By default, each message handler executes within an implicit transaction that commits automatically when the handler returns. The AEP Engine ensures that:

* Store changes are persisted or replicated
* Outbound messages are queued for sending
* The inbound message is acknowledged
* Consensus is established with cluster members (for Event Sourcing)

For most applications, this automatic transaction management is sufficient. However, Talon provides transaction control APIs for specialized scenarios.

## Transaction Control Capabilities

### [Using Savepoints](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions/using-savepoints)

Savepoints allow you to commit work incrementally within a long-running handler. This is useful for:

* Processing large batches where you want to commit progress periodically
* Handlers that may fail partway through and need to avoid reprocessing already-completed work
* Reducing transaction replay time during recovery

When you set a savepoint, Talon commits all work up to that point, including store changes and outbound messages. If the handler subsequently fails, it will resume from the last savepoint rather than replaying from the beginning.

## When to Use Transaction Controls

Consider using transaction controls when:

* **Long-running handlers**: Your handler processes large batches or performs extensive computation
* **Partial failure recovery**: You want to commit intermediate results to avoid reprocessing on failure
* **Memory constraints**: Committing incrementally reduces the amount of buffered state

{% hint style="warning" %}
**Use with Caution**: Transaction controls are advanced features. Most applications should rely on Talon's automatic transaction management. Only use these controls when you have specific requirements that cannot be met by the default behavior.
{% endhint %}

## See Also

* [Transactions](/talon/concepts-and-architecture/transactions) - Conceptual overview of how Talon transactions work
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - How consensus is established
* [Configuring Adaptive Batching](/talon/developing-applications/configuring-the-runtime/transactions/adaptive-batching) - Runtime configuration for transaction batching


# Using Savepoints

The AEP engine allows a microservice to create *savepoints* in a message handler and then roll back to a savepoint to undo changes made between the savepoint and the point of initiation of the rollback. Changes that are rolled back are state changes and outbound sends made by the microservice during that period. The following example illustrates how this is done

```java
AepMessageSender sender;
AepEngine engine;
 
@EventHandler
public void onNewOrder(NewOrderMessage newOrder, MyAppState state) {
  // create a savepoint
  int beforeProcessing = engine.createTransactionSavePoint();
 
 
  Customer customer = state.getCustomers().getCustomer(newOrder.getCustomerId());
 
  // update customer order count 
  customer.setOrdersReceived(customer.getOrdersReceived() + 1);
   
  // on the customer's thousandth order send them a promotion
  if(customer.getOrdersReceived() == 1000) {
    sender.sendMessage("customer-promotions", createPromotionMessage(customer, order));
  }
 
  Product product = state.getProducts(order.getProductId());
  if(product.getItemsAvailable() > newOrder.getQuantity()) {
     product.setItemsAvailable(product.getItemsAvailable() - newOrder.getQuantity());
     messageSender.sendMessage("order-accepted", prepareOrderAccept(newOrder));
  } 
  else {
    //uh-oh, guess we shouldn't have sent that promotion!
    engine.rollbackToSavepoint(beforeProcessing);
    customer.setOrdersRejected(customer.getOrdersRejected() + 1);
    messageSender.sendMessage("order-rejected", prepareOrderReject(newOrder));
  }
}
```

Breaking down the above example we see:

* The microservice creates a savepoint at the beginning of its handler
* It then updates the count of orders received for a customer and possibly sends out a promotional message.
* Later, if the handler determines that there isn't enough inventory to satisfy the order it rolls back to the initial savepoint which:
  * Resets the customer's ordersReceivedCount to its previous value
  * Cancels the possible promotional message for the customer.
* After the rollback, the handler then sends an order rejected message and increments a rejection count for the customer.
* The net result of processing is thus an incremented orderRejectedCount for the customer and an order rejected message.

The sections below discuss savepoints and rollback in more depth.

### Creating Savepoints <a href="#workingwithtalontransactions-creatingsavepoints" id="workingwithtalontransactions-creatingsavepoints"></a>

A microservice can create a savepoint at any point in a message handler by calling the AEP engine's [`createTransactionSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#createTransactionSavepoint\(\))method. The returned savepoint number can then later be reused to rollback to the state at the time the savepoint was created.

### Getting the current savepoint <a href="#workingwithtalontransactions-gettingthecurrentsavepoint" id="workingwithtalontransactions-gettingthecurrentsavepoint"></a>

A microservice can retrieve the current savepoint via the AEP engine's [`getTransactionSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#getTransactionSavepoint\(\)) method.

### Rolling Back to a Savepoint <a href="#workingwithtalontransactions-rollingbacktoasavepoint" id="workingwithtalontransactions-rollingbacktoasavepoint"></a>

This AEP engine's[`rollbackToTransactionSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#rolllbackToTransactionSavepoint\(\)) method rolls back state changes to AEP managed state and any outbound message since the given savepoint (including work in savepoints created after the specified savepoint). The rollback operation leaves the provided savepoint marker in place. For example, if the microservice calls rollback with a savepoint value of 1, a subsequent call to [getTransactionSavepoint()](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#getTransactionSavepoint\(\)) will return 1. New work done after the rollback can thus be rolled back to the same point. Any savepoints after the provided savepoint are discarded. If rollback is called with a savepoint value of 1 when [getTransactionSavepoint()](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#getTransactionSavepoint\(\)) is at 2, savepoint 2 is discarded.

It is worth noting that savepoint rollbacks do not rollback the actual processing of a message from the engine's standpoint, just the effects of its processing. If a handler rolls back all processing work (e.g. [rollbackToTransactionSavepoint(0)](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#rollbackToTransactionSavepoint\(\)), the engine will still consider the message as successfully processed and acknowledge it.

In addition to the restrictions outlined below an attempt to roll back to a savepoint less than 0 or greater than the current savepoint will result in an IllegalStateException.

#### Rollback Errors <a href="#workingwithtalontransactions-rollbackerrors" id="workingwithtalontransactions-rollbackerrors"></a>

An `EAepRollbackError` thrown from this method indicates that there was an internal or runtime error performing the rollback. In this case, microservice event handlers must allow the error to be thrown back for Talon to handle as the microservice store may be in a corrupt state. If the AEP engine can recover by rolling back the entire transaction, the error will be handled according to the[`AppExceptionHandlingPolicy`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.AppExceptionHandlingPolicy.html)`.` Otherwise, the engine will stop with the [EAepRollbackError](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/EAepRollbackError.html).

#### Outbound Message Considerations <a href="#workingwithtalontransactions-outboundmessageconsiderations" id="workingwithtalontransactions-outboundmessageconsiderations"></a>

Outbound messages that are rolled back cannot be reused - the transfer of ownership to the AEP engine is preserved. Additionally, it should be noted that rollback does not rollback changes made to outbound messages' fields. When the engine is configured to dispose on send the engine may dispose of such messages during rollback, so microservices should not rely on messages transferred to the engine being valid post rollback.

#### State and Embedded Entity Considerations <a href="#workingwithtalontransactions-stateandembeddedentityconsiderations" id="workingwithtalontransactions-stateandembeddedentityconsiderations"></a>

Objects that were created since the savepoint that is rolled back may be discarded and cleared by the engine during rollback. Therefore microservices should not attempt to reuse any state objects created since the savepoint that was rolled back.

### Savepoints and Adaptive Batching <a href="#workingwithtalontransactions-adaptivebatchingconsiderations" id="workingwithtalontransactions-adaptivebatchingconsiderations"></a>

When the engine is configured for adaptive batching, multiple inbound messages are grouped into a single transaction. Savepoints don't span multiple inbound messages. Instead, the processing effects of previous messages are fenced off from subsequent savepoints. Effectively, under the covers, the engine creates an internal savepoint for fully processed inbound messages and resets the application visible savepoint to 0 for subsequent messages.

### Savepoints and Multiple Event Handlers <a href="#workingwithtalontransactions-multipleeventhandlers" id="workingwithtalontransactions-multipleeventhandlers"></a>

If there are multiple event handlers for a given event, savepoints *do* span those handlers meaning that a subsequent handler for an event *can* rollback work done by a previous handler. Microservices may create a savepoint via [`createTransactionSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#createTransactionSavepoint\(\)) at the beginning of message processing to avoid rolling back work done by another handler. This allows microservices that chain multiple event handlers together to perform processing a mechanism by which later processors in the chain can completely rollback work.

### Savepoint Restrictions <a href="#workingwithtalontransactions-restrictions" id="workingwithtalontransactions-restrictions"></a>

Transaction savepoints operations (create, get, rollback) are only supported when:

* the engine is configured with savepoints enabled i.e. the AEP engine's[`getEnableTransactionSavepoints()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngineDescriptor.html#getEnableTransactionSavepoints\(\)) method returns `true`
* the engine is backed by a store - [`getStore()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#getStore\(\)) `!= null` - as the AepEngine relies on the store's transaction machinery to perform a rollback.
* called from within application event handlers i.e., only the engine's message processing thread may work with savepoints.
* the engine is not configured for parallel cluster replication i.e. the AEP engine descriptor's[`getReplicateInParallel()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngineDescriptor.html#getReplicateInParallel\(\)) method returns `false`.

If any of the above criteria is not met an `IllegalStateException` is thrown. In addition to the above restrictions a microservice must not use the following savepoint operations in the underlying store:

* [`IStoreBinding.createSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBinding.html#createSavepoint\(\))
* [`IStoreBinding.rollback(int)`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBinding.html#rollback\(int\))
* [`IStoreBinding.rollback()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBinding.html#rollback\(\))
* [`IStoreBinding.getLastSavepoint()`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/ods/IStoreBinding.html#getLastSavepoint\(\))

### Savepoints and HA <a href="#workingwithtalontransactions-haconsiderations" id="workingwithtalontransactions-haconsiderations"></a>

#### State Replication

Savepoint operations can only be performed in a message handler. When using StateReplication this means that savepoint creation and rollbacks can only be done on a Primary instance.

#### Event Sourcing

When using EventSourcing, message handlers are invoked on a Backup instance or an instance recovering from a transaction log. It is *crucial* that a backup or recovering instance's behavior or it will lead to divergence in the application's outbound messages. This means that application logic on a backup must create the same savepoints as a primary and rollback based on the same criteria. For this reason, it is often preferable for an EventSourcing microservice that encounters an error to simply throw an exception from its event handler and let the inbound message's fate be governed by the `AppExceptionHandlingPolicy`.


# Sending Messages

Messages are sent over message bus channels in a fire-and-forget fashion - the microservice can rely on the underlying AEP Engine to deliver the message according to the quality of service configured for a channel. A microservice views message channels as a logical, ordered conduit over which messages flow.

## Overview

The diagram below depicts the path of a message sent through an engine. The basic flow of sending is as follows:

1. The application creates a message
2. The application calls send, passing the message and channel name
3. The engine looks up the channel and uses it to resolve the physical destination on which the message will be sent via the configured channel key
4. The engine queues the message for sending until the associated state changes for the message handler are stabilized to the microservice's store (e.g., replication and transaction log write)
5. Once the microservice's store changes have been stabilized, the enqueued message is then sent. This involves the following sub-steps:
   1. The message is serialized
   2. The serialized contents and message metadata are then packaged in a message bus provider specific message
   3. The message is sent through the underlying transport

### Basic Example

To send a message, the microservice needs an [`AepMessageSender`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepMessageSender.html) which can be injected via the `@AppInjectionPoint` annotation:

```java
public class OrderProcessor {
    private AepMessageSender messageSender;

    @AppInjectionPoint
    public void setMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    @EventHandler
    public void onNewOrder(NewOrderMessage message) {
        // Create acknowledgment message
        OrderAckMessage ack = OrderAckMessage.create();
        ack.setOrderId(message.getOrderId());
        ack.setStatus("ACCEPTED");

        // Send on the 'order-acks' channel
        messageSender.sendMessage("order-acks", ack);
    }
}
```

The `AepMessageSender` provides several send methods:

| Method                                                                     | Description                                   |
| -------------------------------------------------------------------------- | --------------------------------------------- |
| `sendMessage(String channel, MessageView message)`                         | Sends a message on the named channel          |
| `sendMessage(String bus, String channel, MessageView message)`             | Sends a message on a specific bus and channel |
| `sendMessage(String bus, String channel, MessageView message, String key)` | Sends with a caller-provided key (advanced)   |

## Creating Messages For Send

Message types generated by ADM have no public constructors. Instances are created via the static `create()` factory method on the message class:

```java
OrderAckMessage ack = OrderAckMessage.create();
ack.setOrderId(message.getOrderId());
ack.setStatus("ACCEPTED");
```

By routing all message creation through `create()` factory methods, Talon can transparently switch between pooled and non-pooled allocation of messages without requiring changes to business logic. This enables [zero-garbage operation](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage) for latency-sensitive applications.

### Populating Messages

Once created, messages can be populated using setter methods just like any other POJO:

```java
@EventHandler
public void onNewOrder(NewOrderMessage message) {
    // Create and populate the message
    OrderAckMessage ack = OrderAckMessage.create();
    ack.setOrderId(message.getOrderId());
    ack.setTimestamp(System.currentTimeMillis());
    ack.setStatus("ACCEPTED");

    // Send the message
    messageSender.sendMessage("order-acks", ack);
}
```

For advanced encoding-specific population mechanisms (e.g., Xbuf), see [Coding for Zero Garbage](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/coding-for-zero-garbage).

### Disposing Unsent Messages

If a message is created but not sent, the application should call `dispose()` on the message to return it to the pool:

```java
OrderAckMessage ack = OrderAckMessage.create();
ack.setOrderId(orderId);

// Validation fails - don't send
if (!isValid(ack)) {
    ack.dispose();  // Return to pool
    return;
}

messageSender.sendMessage("order-acks", ack);
```

Failing to dispose of unsent messages will not cause memory leaks, but will prevent subsequent `create()` calls from reusing pooled objects, potentially impacting allocation rates.

{% hint style="warning" %}
Once a message is sent, ownership transfers to the engine and the application must not modify or access the message. See [Programming Fundamentals](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/programming-fundamentals#outbound-messages) for details on message lifecycle and ownership.
{% endhint %}

## Message Keys and Topic Resolution

When a message is sent, the engine resolves the physical topic or destination using the channel's configured **message key**. The message key can contain static values and dynamic variables that are resolved from the message being sent.

### Static Keys

A static key does not contain variable substitutions:

```xml
<channel name="heartbeats">
  <key>APP/HEARTBEATS</key>
</channel>
```

### Dynamic Keys with Variable Substitution

Variable portions of a key are denoted using `${variableName}` syntax:

```xml
<channel name="order-events">
  <key>ORDERS/${region}/${orderType}</key>
</channel>
```

When a message is sent, the engine resolves variables by:

1. Looking up the value on the message (e.g., `message.getRegion()`)
2. Using values from a configured key resolution table
3. Using the default value if specified in the key

### Message-Based Key Resolution

The most common approach is to resolve key variables from fields on the message:

```xml
<channel name="order-events">
  <key>ORDERS/${region}/${orderType}</key>
</channel>
```

When sending an `OrderEventMessage` with:

* `region = "US"`
* `orderType = "MARKET"`

The resolved topic would be: `ORDERS/US/MARKET`

### Key Resolution Tables

For variables not present on the message, applications can configure a key resolution table:

**Configuration-Based Resolution Table**:

```xml
<app name="order-processor">
  <messaging>
    <buses>
      <bus name="order-bus">
        <channels>
          <channel name="order-events" join="false">
            <keyResolutionTable>
              <entry>
                <key>region</key>
                <value>US</value>
              </entry>
              <entry>
                <key>shard</key>
                <value>1</value>
              </entry>
            </keyResolutionTable>
          </channel>
        </channels>
      </bus>
    </buses>
  </messaging>
</app>
```

**Programmatic Resolution Table**:

```java
@EventHandler
public void onChannelUp(AepChannelUpEvent event) {
    MessageChannel channel = event.getMessageChannel();

    // Properties-based table
    Properties krt = new Properties();
    krt.setProperty("region", "EMEA");
    krt.setProperty("shard", "1");
    channel.setKeyResolutionTable(krt);
}
```

**Raw (Zero Garbage) Resolution Table**:

For performance-sensitive applications using Xbuf encoding:

```java
private final RawKeyResolutionTable globalKeyResolutionTable =
    RawKeyResolutionTable.create(64);

@AppEventHandlerInit
public void init() {
    krt.put("Region", XString.create("EMEA"));
    krt.put("Shard", XString.create("1"));
}

@EventHandler
public void onOrderEventChannelUp(AepChannelUpEvent channelUpEvent) {
    channelUpEvent.getMessageChannel()
                  .setRawKeyResolutionTable(globalKeyResolutionTable);
}
```

When using a `RawKeyResolutionTable`:

* The table should not be modified after being set on a channel
* XString values must not be modified after insertion
* Cannot use both Raw and Properties-based tables on the same channel

### Default Values

Variables can specify default values using the syntax `${variableName:defaultValue}`:

```xml
<channel name="order-events">
  <key>ORDERS/${region:US}/${orderType:LIMIT}</key>
</channel>
```

If `region` cannot be resolved from the message or key resolution table, "US" will be used.

### Channel Key Functions

Channel key functions enable dynamic computation of key portions at runtime. Functions are declared using `#[variableName = functionName(arg1, argN)]` syntax:

```xml
<channel name="order-events">
  <key>ORDERS/#[shard=hash(${orderId}, 4)]</key>
</channel>
```

#### Built-In Functions

| Function | Description                    | Arguments                                                                        |
| -------- | ------------------------------ | -------------------------------------------------------------------------------- |
| `hash`   | Hashes value into N partitions | <p>Arg1: value to hash<br>Arg2: number of partitions<br>Returns: 1 through N</p> |
| `env`    | Looks up runtime property      | <p>Arg1: property name<br>Arg2 (optional): default value</p>                     |
| `concat` | Concatenates two strings       | <p>Arg1: string1<br>Arg2: string2</p>                                            |

**Hash Function Example**:

```xml
<buses>
  <bus name="order-bus" descriptor="solace://solhost:55555">
    <channels>
      <channel name="new-orders">
        <key>NEWORDERS/#[ordershard=hash(${orderId}, 3)]</key>
      </channel>
    </channels>
  </bus>
</buses>

<apps>
  <app name="sender">
    <messaging>
      <bus name="order-bus">
        <channels>
          <channel name="new-orders" join="false"/>
        </channels>
      </bus>
    </messaging>
  </app>

  <app name="receiver-1">
    <messaging>
      <bus name="order-bus">
        <channels>
          <channel name="new-orders" join="true">
            <filter>ordershard=1|2</filter>
          </channel>
        </channels>
      </bus>
    </messaging>
  </app>
</apps>
```

In this example:

* The sender sends on topics: `NEWORDERS/1`, `NEWORDERS/2`, or `NEWORDERS/3`
* The receiver subscribes to: `NEWORDERS/1` and `NEWORDERS/2` (but not 3)

#### Custom Channel Key Functions

Applications can define custom channel key functions by setting `nv.sma.channelkeyfunctioncontainers` to a comma-separated list of classes containing channel key functions.

**Requirements for Custom Functions**:

* Must be declared `public static`
* First argument must be the key being resolved (to which the function appends)
* May accept additional `XString` arguments representing function arguments from the channel key

**Example**:

Configuration:

```xml
<env>
  <nv>
    <sma>
      <channelkeyfunctioncontainers>
        com.example.CustomChannelKeyFunctions
      </channelkeyfunctioncontainers>
    </sma>
  </nv>
</env>
```

Implementation:

```java
public class CustomChannelKeyFunctions {
    /**
     * A custom key function that substitutes the first character
     * of the provided value into the message key.
     */
    public static final void abbreviate(final XString messageKey,
                                       XString value) {
        messageKey.append(value.charAt(0));
    }
}
```

## Message Key Validation

When resolving topics dynamically from message fields, it's important to validate that values don't introduce illegal characters or empty topic levels. Configure validation using these properties:

| Property                        | Default | Description                                                                                                                                                                  |
| ------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| nv.sma.maxresolvedkeylength     | 0       | Maximum resolved key length. When > 0, validates key length. Set to lowest max length across all bindings for portability.                                                   |
| nv.sma.cleanmessagekey          | false   | When true, replaces non-alphanumeric characters in variable values with underscore. For example, `"Asia/Pac"` becomes `"Asia_Pac"`. Does not apply to channel key functions. |
| nv.sma.allowemptykeyfield       | false   | When false, empty string values in key variables cause resolution to fail. Does not apply to channel key functions.                                                          |
| nv.sma.treatemptykeyfieldasnull | false   | When true, treats empty strings as null, allowing fallback to other sources or default values. Takes precedence over `allowemptykeyfield`.                                   |
| nv.sma.validatemessagekey       | false   | When true, enables message key validation before sending.                                                                                                                    |

## Caller-Provided Keys

Applications can provide the resolved key directly in the send call:

```java
@EventHandler
public void onNewOrder(NewOrderMessage message) {
    OrderEventMessage event = OrderEventMessage.create();
    event.setOrderId(message.getOrderId());

    // Provide custom topic directly
    messageSender.sendMessage("order-processing-bus",
                             "order-event-channel",
                             event,
                             "CUSTOMORDEREVENTS/" + message.getOrderType());
}
```

When using caller-provided keys:

* Variable substitution is **not** performed on the provided key
* Key validation **is** performed
* The key will be prefixed with the channel name if `topic_starts_with_channel=true`

{% hint style="warning" %}
Caller-provided keys should be used with care as they break the ability to change topics at runtime and receivers won't know how to subscribe to such sends.
{% endhint %}

## Message Sequencing

The AEP Engine assigns monotonically increasing sequence numbers to outbound messages to enable receivers to detect message loss or out-of-order delivery. Sequence numbers are managed per bus+channel+qos combination.

### Enabling Sequence Numbers

Sequence numbers are enabled per channel using the `<sequenceMessages>` element:

```xml
<app name="order-processor">
  <messaging>
    <buses>
      <bus name="order-bus">
        <channels>
          <channel name="order-events" join="false">
            <sequenceMessages>true</sequenceMessages>
          </channel>
        </channels>
      </bus>
    </buses>
  </messaging>
</app>
```

### Solicited vs Unsolicited Sends

By default, sequence numbering behavior differs between solicited and unsolicited sends:

**Solicited sends** (in-transaction sends):

* Sequence numbers **are** set by default
* Controlled by `setOutboundSequenceNumbers` (default: true)

**Unsolicited sends** (out-of-transaction sends):

* Sequence numbers are **not** set by default
* Controlled by **both** `setOutboundSequenceNumbers` (default: true) **and** `sequenceUnsolicitedSends` (default: false)
* To enable sequence numbers on unsolicited sends, set `sequenceUnsolicitedSends=true`

**Configuration example**:

```xml
<app name="gateway-app">
  <messaging>
    <!-- messaging config -->
  </messaging>

  <!-- Enable sequence numbers on unsolicited sends -->
  <sequenceUnsolicitedSends>true</sequenceUnsolicitedSends>

  <!-- Global control for all outbound sequence numbers (default: true) -->
  <setOutboundSequenceNumbers>true</setOutboundSequenceNumbers>
</app>
```

{% hint style="info" %}
When using `sequenceUnsolicitedWithSolicitedSends=true` for concurrent send safety, unsolicited sends are converted to solicited sends and thus follow solicited send sequencing behavior (controlled only by `setOutboundSequenceNumbers`). See [Concurrent Sends](#concurrent-sends) for details.
{% endhint %}

### Accessing Sequence Numbers

Applications can access sequence numbers from received messages:

```java
@EventHandler
public void onOrderEvent(OrderEventMessage message) {
    long seqNo = message.getMessageMetadata().getSequenceNumber();
    // Check for gaps or out-of-order delivery
}
```

## Unsolicited Sends

An **unsolicited send** is a message sent from outside of a message handler - not in response to a received message. Unsolicited sends are common in gateway applications that input messages from external sources.

### Overview

When messages are sent from within a message handler (solicited sends), the AEP Engine automatically manages send stability and ensures zero-loss delivery across failovers. For unsolicited sends, additional configuration and application participation may be required to ensure zero-loss delivery.

### Send Stability Tracking

The AEP Engine provides [`AepSendStabilityEvent`](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepSendStabilityEvent.html) to notify applications when an unsolicited send has been stabilized and is guaranteed to be delivered. This allows gateway applications to track which messages have been successfully delivered.

**Enable Send Stability Events**:

```xml
<app name="file-gateway" mainClass="com.example.FileGateway">
  <messaging>
    <!-- ... -->
  </messaging>
  <dispatchSendStabilityEvents>true</dispatchSendStabilityEvents>
  <sequenceUnsolicitedWithSolicitedSends>true</sequenceUnsolicitedWithSolicitedSends>
</app>
```

### Send Stability Semantics

An unsolicited send is considered stabilized according to the microservice's [`InboundEventAcknowledgementPolicy`](/talon/reference/configuration#policy-configuration):

**Sent in transaction stream** (`sequenceUnsolicitedWithSolicitedSends=true`):

* **OnSendStability**: Stabilized when all messages in the transaction (and prior transactions) are acknowledged
* **OnStoreStability**: Stabilized when the transaction has been replicated to backup and/or transaction log

**Sent outside transaction stream** (`sequenceUnsolicitedWithSolicitedSends=false` and `replicateUnsolicitedSends=false`):

* Stabilized on receipt of acknowledgment from the message bus

### Correlating Send Stability

Applications need to correlate stability events with the source data that triggered the send. Use one of these approaches:

**1. Use Message Fields**:

```java
@EventHandler
public void onSendStability(AepSendStabilityEvent event) {
    FileLineMessage message = (FileLineMessage) event.getMessage();
    long lineNumber = message.getLineNumber();
    // Update cursor file, acknowledge upstream, etc.
}
```

**2. Use Message Attachments**:

When correlation data shouldn't be in the message payload:

```java
// During send
FileLineMessage message = FileLineMessage.create();
message.setText(line);
message.setAttachment(new Long(lineNumber));
messageSender.sendMessage("line-input", message);

// On stability
@EventHandler
public void onSendStability(AepSendStabilityEvent event) {
    FileLineMessage message = (FileLineMessage) event.getMessage();
    long lineNumber = (Long) message.getAttachment();
    // Process stability for this line number
}
```

### Example: File Gateway with Send Stability

The following example shows a file gateway that reads lines from a file and publishes each line. It tracks send stability to maintain a cursor file:

```java
@AppHAPolicy(value = HAPolicy.StateReplication)
public class FileInputGateway {
    Tracer tracer = Tracer.create("file-tailer", Level.INFO);
    private volatile AepMessageSender messageSender;
    private volatile AepEngine engine;

    @Configured(property = "gateway.filename")
    private volatile String filename;

    private RandomAccessFile cursor;
    private volatile long stabilizedLine = 0;

    @AppInjectionPoint
    public void injectMessageSender(AepMessageSender messageSender) {
        this.messageSender = messageSender;
    }

    @AppInjectionPoint
    public void injectAepEngine(AepEngine engine) {
        this.engine = engine;
    }

    @AppMain
    public void run(String[] args) throws Exception {
        BufferedReader reader = new BufferedReader(new FileReader(filename));
        try {
            cursor = new RandomAccessFile(filename + ".cursor", "rwd");
            cursor.setLength(8);
            cursor.seek(0);

            // Wait for messaging to start
            engine.waitForMessagingToStart();

            // Read and send lines
            String line = null;
            stabilizedLine = cursor.readLong();
            int lineNumber = 0;

            while ((line = reader.readLine()) != null) {
                if (++lineNumber <= stabilizedLine) {
                    continue;
                }

                FileLineMessage message = FileLineMessage.create();
                message.setText(line);
                message.setFileName(filename);
                message.setLineNumber(lineNumber);
                message.setAttachment(new Long(message.getLineNumber()));
                messageSender.sendMessage("line-input", message);
            }

            // Wait for send stability before exiting
            while (stabilizedLine < lineNumber) {
                Thread.sleep(100);
            }
        }
        finally {
            reader.close();
            cursor.close();
        }
    }

    @EventHandler
    public final void onSendStability(AepSendStabilityEvent event)
            throws IOException {
        FileLineMessage message = (FileLineMessage) event.getMessage();
        stabilizedLine = (Long) message.getAttachment();
        cursor.seek(0);
        cursor.writeLong(stabilizedLine);
        cursor.getFD().sync();
        tracer.log("Got stability for: " + stabilizedLine, Level.INFO);
    }

    @AppStat(name = "Lines Sent")
    public long getLinesSent() {
        return stabilizedLine;
    }
}
```

Configuration:

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-ddl">
  <buses>
    <bus name="file-connector">
      <provider>activemq</provider>
      <address>localhost</address>
      <port>61616</port>
      <channels>
        <channel name="line-input">
          <qos>Guaranteed</qos>
          <key>${filename}</key>
        </channel>
      </channels>
    </bus>
  </buses>

  <apps>
    <app name="file-input-gateway"
         mainClass="com.sample.FileInputGateway">
      <messaging>
        <factories>
          <factory name="com.sample.messages.FileGatewayMessageFactory"/>
        </factories>
        <buses>
          <bus name="file-connector">
            <channels>
              <channel name="line-input" join="false"/>
            </channels>
          </bus>
        </buses>
      </messaging>
      <dispatchSendStabilityEvents>true</dispatchSendStabilityEvents>
      <sequenceUnsolicitedWithSolicitedSends>true</sequenceUnsolicitedWithSolicitedSends>
    </app>
  </apps>
</model>
```

### Send Exception Handling

Exceptions can occur during send calls or asynchronously via negative acknowledgments (nacks) from the messaging provider. Handling depends on configuration:

**Sent outside transaction stream**:

* **Send Exception**: Thrown to caller, no `AepSendStabilityEvent` dispatched
* **Nack**: Reported in `AepSendStabilityEvent` via non-null `getStatus()`

**Sent in transaction stream**:

Exception handling depends on [`InboundEventAcknowledgementPolicy`](/talon/reference/configuration#policy-configuration) and [`MessageSendExceptionHandlingPolicy`](/talon/reference/configuration#policy-configuration):

**OnSendStability**:

* Send failure with `LogExceptionAndContinue`: Exception logged and reported in `AepSendStabilityEvent`
* Send failure with `TreatAsFatal`: Engine shuts down, exception in `AepEngineStoppedEvent`
* Nack: Engine shuts down, exception in `AepEngineStoppedEvent`

**OnStoreStability**:

* `AepSendStabilityEvent` dispatched after replication
* Send failure with `LogExceptionAndContinue`: Exception logged (not in stability event as it was already dispatched)
* Send failure with `TreatAsFatal`: Engine shuts down
* Nack: Engine shuts down

### HA Considerations for Unsolicited Sends

* **EventSourcing**: Unsolicited sends are not replicated or persisted in Event Sourcing mode. Use State Replication for unsolicited sends requiring durability.
* **Failover**: Engine does not dispatch `AepSendStabilityEvent` for recovered sends after failover
* **At Least Once Delivery**: Unsolicited sends provide at-least-once delivery semantics. Downstream applications must handle potential duplicates.

### Additional Considerations

* **Threading**: `AepSendStabilityEvent` is dispatched on the engine's multiplexer thread. Applications performing expensive or blocking operations should use a separate thread.
* **Event Ordering**:
  * With `sequenceUnsolicitedWithSolicitedSends=true`: Stability events issued in send order
  * With `sequenceUnsolicitedWithSolicitedSends=false`: Stability events issued in acknowledgment order (usually ordered per bus+channel+qos)

## Concurrent Sends

The Talon runtime ensures single-threaded operation for solicited sends (in-transaction sends). However, unsolicited sends may introduce concurrency that requires additional configuration when:

* Multiple unsolicited sends are performed concurrently using multiple threads, or
* Unsolicited sends are performed concurrently with solicited sends

### Thread Safety Configuration

Talon provides two alternative mechanisms for ensuring concurrent send safety:

#### Option 1: Bus-Level Concurrent Sends (Recommended)

Configure the message bus binding for concurrent sends using the `enable_concurrent_sends` connection descriptor property:

```xml
<buses>
  <bus name="sample-bus" descriptor="activemq://localhost:61616?enable_concurrent_sends=true">
    <channels>
      <!-- channel configuration -->
    </channels>
  </bus>
</buses>
```

This is the preferred, modern approach applicable to all message bus bindings supported by Talon. It ensures correct operation for both concurrent send scenarios:

* Multiple unsolicited sends performed concurrently using multiple threads
* Unsolicited sends performed concurrently with solicited sends

#### Option 2: Transaction Pipeline Integration (Legacy)

Use the `sequenceUnsolicitedWithSolicitedSends` configuration parameter:

```xml
<model>
  <buses>
    <bus name="sample-bus" descriptor="activemq://localhost:61616">
      <channels>
        <!-- channel configuration -->
      </channels>
    </bus>
  </buses>

  <apps>
    <app name="sample-app" mainClass="com.sample.SampleApp">
      <messaging>
        <!-- messaging config -->
      </messaging>

      <!-- Alternative: inject unsolicited sends into transaction pipeline -->
      <sequenceUnsolicitedWithSolicitedSends>true</sequenceUnsolicitedWithSolicitedSends>
    </app>
  </apps>
</model>
```

This legacy approach injects unsolicited sends into the engine's transaction pipeline, effectively converting them to solicited sends. This achieves concurrent send safety as a side effect of transaction pipeline integration.

**Key difference between the two approaches**:

The approaches differ in how sequence numbers are applied to unsolicited sends. See [Message Sequencing](#message-sequencing) for details on sequence number configuration.

**With `enable_concurrent_sends`**:

* Unsolicited sends remain out-of-transaction
* Follow normal unsolicited send sequencing behavior (no sequence numbers by default)

**With `sequenceUnsolicitedWithSolicitedSends`**:

* Unsolicited sends are converted to solicited sends (injected into transaction pipeline)
* Follow solicited send sequencing behavior (sequence numbers set by default)

{% hint style="info" %}
Both `enable_concurrent_sends` and `sequenceUnsolicitedWithSolicitedSends` are alternative mechanisms for achieving concurrent send safety. They are equivalent except for the default behavior of sequence numbering on unsolicited sends (see [Message Sequencing](#message-sequencing)). Setting both is safe but not necessary. The `enable_concurrent_sends` approach is preferred for new applications, while `sequenceUnsolicitedWithSolicitedSends` remains available for applications that prefer the transaction injection approach.
{% endhint %}

### Configuration Summary

The following table summarizes how to configure Talon for concurrent sends in various scenarios:

| Solicited Sends | Concurrent Unsolicited Sends | Sequence Number in Solicited Sends | Sequence Number in Unsolicited Sends | Configuration                                |
| --------------- | ---------------------------- | ---------------------------------- | ------------------------------------ | -------------------------------------------- |
| No              | Yes                          | N/A                                | Yes or No                            | `enable_concurrent_sends=true`               |
| Yes             | Yes                          | Yes                                | No                                   | `enable_concurrent_sends=true`               |
| Yes             | Yes                          | No                                 | Yes                                  | `enable_concurrent_sends=true`               |
| Yes             | Yes                          | No                                 | No                                   | `enable_concurrent_sends=true`               |
| Yes             | Yes                          | Yes                                | Yes                                  | `sequenceUnsolicitedWithSolicitedSends=true` |

{% hint style="info" %}
When using sequence numbers with concurrent sends, Talon ensures that messages are transmitted on the wire in the same sequence as the sequence numbers assigned to them.
{% endhint %}

## See Also

* [Registering Message Interest](/talon/developing-applications/configuring-messaging/registering-message-interest) - Receiving messages
* [Configuring Bus Connections](/talon/developing-applications/configuring-messaging/configuring-bus-connections) - Bus and channel configuration
* [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Message handler best practices
* [Controlling Transactions](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions) - Transaction management


# Unhandled Messages

From Talon's point of view, an unhandled message is a message that is received but not processed by a message handler. This section discusses how such receipt is handled.

Unhandled messages fall into 3 categories:

1. **SMA Undeserializable Messages** - Messages that are received from an underlying SMA [MessageBusBinding](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageBusBinding.html) that cannot be deserialized into a message.
2. **SMA Unsolicited Messages** - Messages that can be successfully deserialized into a message but are on a [MessageChannel](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageChannel.html) that is not joined or not known to the receiving microservice.
3. **AEP Unhandled Messages** - Valid messages intentionally attracted to the microservice on a joined [MessageChannel](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageChannel.html), but for which the microservice has no declared [EventHandler](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/annotations/EventHandler.html).

The first two situations are trapped by SMA and are reported via an SMA [UnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/event/UnhandledMessageEvent.html), the latter situation is trapped by the AepEngine and reported to the microservice as an [AepUnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepUnhandledMessageEvent.html).

## SMA Triggered Unhandled Message Events

Received messages that are not deserializable or unsolicited are trapped by SMA and reported to the AepEngine via an SMA [UnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/event/UnhandledMessageEvent.html) which the AepEngine dispatches to the appropriate [EventHandler](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/annotations/EventHandler.html), if registered.

### Causes <a href="#unhandledmessages-causes" id="unhandledmessages-causes"></a>

Below are some typical causes for SMA [UnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/event/UnhandledMessageEvent.html) events.

* **Receiving a message on a channel that has not been joined:** this usually indicates a configuration or design error in topics or subscriptions. The intent behind the SMA's message channel abstraction is to define named logical conduits between peers. When a message is received on an unjoined channel, it means that messages are being received by the microservice on a channel that the application designer wasn't expecting to receive traffic on.
* **Receiving a message on an unknown channel:** This usually indicates a problem with topic subscriptions in which two channels are overlapping and causing a microservice to attract messages from a sender on another channel.
* **Receiving a message that has no channel or metadata:** Messages sent to X microservice are typically expected to have SMA [MessageMetadata](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageMetadata.html) that can be used identify the message's transmission channel, type and factory. Receipt of a message without metadata for a bus that expects will thus result in an error.
* **Deserialization Errors:** The SMA contract is to pass the message in deserialized MessageView (POJO) form to the AepEngine. If the message's view factory cannot be found or the data is corrupted, it will result in an error.

### UnhandledMessageEvent

The following fields are available on the UnhandledMessageEvent.

| Field                        | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| BackingMessage               | When running in a Talon XVM, this will contain a [SrvMonUnhandledMessageMessage](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/alert/SrvMonUnhandledMessageMessage.html). This is a monitoring alert message that contains this event in a SMA serializable form which allows the microservice to serialize the contents of this event for auditing and administrative purposes.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| MessageBusBinding            | The message bus binding where `UnhandledMessageEvent` originated, or `null` if not available.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| MessageKey                   | The unhandled message's key (e.g. bus destination) on which the message was received (if available and the source binding supports transport of the key).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| MessageMetadata              | The unhandled message's metadata. See the [MessageMetadata](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/MessageMetadata.html) javadoc for a description of these fields.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| MessageSMATransportMessageId | <p>The unhandled message's SMA transport specific message id (if available).</p><p>The SMA transport specific message ID may be supplied by bindings for which there is a notion of a unique identifier for a message. A null value means that either the binding doesn't support the notion of such an id or that the id wasn't available.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| SerializedPayloadBlob        | <p>The serialized payload blob representing the received serialized message that was unhandled.</p><p>The serialized payload is provided with the UnhandledMessageEvent to allow handlers to quarantine (save) the message for subsequent handling by administrators or by tools which allow the unhandled message to be acknowledged upstream. Note, however, that UnhandledMessageEvents are emitted in exceptional cases, and even if the serialized payload is provided, there is no guarantee that it was not corrupted during receipt.</p><p>This field is not guaranteed to be available. Reasons why it may not be available include:</p><ul><li>The payload content was missing from SMA transport level message (poison/corrupt message).</li><li>The SMA binding doesn't serialize messages, and consequently can provide a serialized form.</li><li>There was a bug in the binding around handling the message.</li></ul><p>The encoding type of these bytes is determined by the corresponding metadata, so it is also generally necessary to ensure that that metadata is persisted along with the serialized payload to allow this serialized form to be deserialized.</p> |
| SerializedMetadataBlob       | <p>The serialized metadata blob representing the metadata as received in serialized form.</p><p>The serialized metadata is provided with the UnhandledMessageEvent to allow handlers to quarantine (save) the message for subsequent diagnostic handling in cases where received metadata was corrupted on the wire or during receipt in the binding.</p><p>This field is not guaranteed to be available. Reasons why it may not be available include:</p><ul><li>The payload content was missing from SMA transport level message (poison/corrupt message).</li><li>The SMA binding doesn't serialize messages, and consequently can provide a serialized form.</li><li>There was a bug in the binding around handling the message.</li></ul><p>The encoding type of these bytes is determined by the corresponding metadata, so it is also generally necessary to ensure that the metadata is persisted along with the serialized payload to allow this serialized form to be deserialized.</p>                                                                                                                                                                                         |
| UnwrappedMessage             | The unwrapped (un-deserialized) message. This is the raw transported form of the message which is dependent on the source binding. For example, for a JMS binding this could be a javax.jms.Message, or a buffer when using the native Solace binding.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| Reason                       | A Throwable object describing the reason why the message was unhandled.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |

### Acknowledgement

For unhandled Guaranteed messages, determining whether or not to acknowledge unhandled message is an important consideration.

If the message is not acknowledged:

* The message will be redelivered if the microservice is restarted or fails over to a backup, which can be undesirable.
* If there are a large number of unacknowledged messages, it puts a burden on the messaging provider, which could hit resource limits.
* If the SMA provider doesn't support individual message acknowledgements, then failing to acknowledge the unhandled message can block subsequent acknowledgements.

But acknowledging the message:

* Is dangerous because the message will be discarded without being processed by the microservice.

Prior to the 3.2 release, whether or not the triggering message was acknowledged was left to the SMA bus binding, which in most cases would not acknowledge the message.

As of the 3.2 release, a microservice that declares an [EventHandler](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/annotations/EventHandler.html) for the [UnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/sma/event/UnhandledMessageEvent.html) can control whether or not the triggering message is acknowledged. By default, the platform leans towards ensuring that the message is not discarded and will not acknowledge the message unless instructed to do so by the microservice. A microservice can control acknowledgement as discussed below.

#### Default Acknowledgement Behavior

Auto acknowledgement for unhandled messages can be globally set to enabled by setting the configuration environment property.

```
nv.sma.unhandledmessageevent.autoack=true
```

As of 3.2, setting the above property will cause UnhandledMessageEvents to be acknowledged regardless of whether or not there is a handler for the messages, unless a handler specifically calls setAutoAck(false) on the event. Acknowledgement is done after the event is dispatched in the Aep Engine transaction thread (i.e., the event multiplexer thread).

{% hint style="info" %}
In the 3.1.3 patch release, the Solace binding was patched, allowing this property to control whether the receiving thread will acknowledge the message. When set, the message may be acknowledged prior to the event being dispatched to the application by the aep engine transaction thread
{% endhint %}

{% hint style="warning" %}
Prior to version 3.4.373 this property was named nv.sma.unhandledmessageevent.autoAck. This property was changed to be all lowercase for uniformity with other environment property names. The old camelcase property name is still supported for backwards compatibility, but the newer property name takes precendence. It is therefore important for applications that may specify and override this property from multiple locations (e.g. system property and ddl) use the same case everywhere.
{% endhint %}

#### Auto Acknowledgement

A microservice can explicitly override the default acknowledgement behavior of unhandled messages by declaring an EventHandler for UnhandledMessageEvent and setting the event's autoAck behavior.

```java
@EventHandler
public void onUnhandledMessage(UnhandledMessageEvent event) {
  byte [] toQuarantine = event.getBackingMessage().serializeToByteArray();
  // ... quarantine the above message in some fashion.
   
  // Mark the event for auto acknowledgement, the 
  // engine will acknowledge it.
  event.setAutoAck(true);
}
```

#### Explicit Acknowledgement <a href="#unhandledmessages-explicitacknowledgement" id="unhandledmessages-explicitacknowledgement"></a>

t is also possible to acknowledge the event asynchronously outside of the event handler. This advanced usage is useful for a microservice that might perform a blocking operation processing the UnhandledMessageEvent.

```java
@EventHandler
public void onUnhandledMessage(UnhandledMessageEvent event) {
  // Disable auto acknowledgement:
  event.setAutoAck(false);
  
  // Acquire the event so the platform doesn't return it
  // to a pool:
  event.acquire();
 
  // Asynchronously handle the event:
  Thread thread = new Thread(new Runnable() {
    public void run() {
      byte [] toQuarantine = event.getBackingMessage().serializeToByteArray();
      // ... quarantine the above message in some fashion that might block.
      try {
        event.acknowledge();
      }
      catch (SmaException e) {
        e.printStackTrace();
      }
      finally {
        event.dispose();
      }
    }
  }
}
```

## AEP Triggered Unhandled Message Events

An [AepUnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepUnhandledMessageEvent.html) is emitted when a message is successfully dispatched to an AepEngine from an SMA binding, but no application EventHandler is found by the AepEngine.

### Causes <a href="#unhandledmessages-causes.1" id="unhandledmessages-causes.1"></a>

* The application is incorrectly configured to join a Message Channel that it shouldn't have joined.
* The application's channel filter is not narrow enough and some message types are coming though that shouldn't be.
* A publisher is publishing messages to the channel that it shouldn't be publishing.

An [AepUnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepUnhandledMessageEvent.html) doesn't necessarily indicate a serious problem, but the presence of such events at the very least indicates inefficiencies in the type of messages that are being attracted by the applications subscriptions.

### Acknowledgement <a href="#unhandledmessages-acknowledgement.1" id="unhandledmessages-acknowledgement.1"></a>

AEP unhandled messages are acknowledged like a normally handled message, except they result in an empty transaction with external effects. The message is acknowledged when the transaction is stabilized. For a microservice using EventSourcing that is using a disk based persister, such messages will end up in the microservice's recovery transaction log, and, consequently, will not be lost in the event of acknowledgement. For a StateReplication microservice or an EventSourcing microservice without a Store persister, an application should ensure that the message is appropriately quarantined prior to returning from the [AepUnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepUnhandledMessageEvent.html) event handler and is acknowledged.

{% hint style="info" %}
Some Additional Notes:

* Because [AepUnhandledMessageEvent](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/event/AepUnhandledMessageEvent.html)s are part of a servicd's transaction pipeline, Talon doesn't provide the capability for asynchronous acknowledgement.
* For StateReplication microservices, AEP unhandled messages will be logged in the inbound message log, but this log is not synced to disk as part of transaction commit. This means that applications should not rely on it for quarantining purposes.
  {% endhint %}


# Message Injection

This section covers how to programmatically inject messages into your microservice for processing, including scheduled and one-time injection.

## Overview

Message injection allows you to create and process messages programmatically without receiving them from external sources. This is useful for:

* **Timers and Periodic Tasks** - Execute business logic on a schedule
* **State Initialization** - Bootstrap your store with initial data
* **Workflow Orchestration** - Trigger multi-step processes

Injected messages are processed exactly like externally-received messages: they're dispatched to handlers and execute within transactions with full consensus guarantees.

## Injection Patterns

* **One-Time Injection** - Inject a single message for immediate processing
* **Scheduled Injection** - Schedule messages to execute at specific times or intervals
* **Initialization Messages** - Special handling for first/initial messages during startup

## Topics

* [**Injecting Messages**](/talon/developing-applications/authoring-user-code/message-injection/injecting-messages) - Programmatically create and inject messages for processing
* [**Scheduling Messages**](/talon/developing-applications/authoring-user-code/message-injection/scheduling-messages) - Schedule messages for future or periodic execution

## Related Topics

* [Initializing the Microservice](/talon/developing-applications/authoring-user-code/lifecycle/initializing-the-microservice) - Use first/initial messages for initialization
* [Message Processing](/talon/developing-applications/authoring-user-code/message-processing) - How injected messages are processed


# Injecting Messages

## Overview <a href="#schedulingandinjectionofmessages-overview" id="schedulingandinjectionofmessages-overview"></a>

At the core of the AEP Engine is its event multiplexer - a prioritized dispatch loop that pumps messages into your microservice’s handlers. Whether a message originates from the prestart event, arrives via subscriptions on the underlying bus or explicitly injected by the user, it’s enqueued by priority in the multiplexer. The multiplexer then dequeues each message in turn and routes it to the appropriate handler. All messages and events are dispatched by the engine multiplexer's dispatcher thread.

This section covers how to inject messages directly into the multiplexer - a technique you can use to schedule deferred work from within an event handler or to have an external thread enqueue tasks into your microservice.

### Injection Thread

Any thread can inject messages into the engine's multiplexer.

## Message Injection <a href="#schedulingandinjectionofmessages-injectingmessages" id="schedulingandinjectionofmessages-injectingmessages"></a>

There are two variants of message injection

* Prioritized Injection with immediate dispatch
* Injection with delayed dispatch

All injection is done using an overloaded variant of the AepEngine's `injectMessage()` method.

### Injection with Immediate Dispatch

#### Normal Injection

Normal injection is injection with priority 0. The following code illustrates how to perform injection with normal (0) priority for immediate dispatch.

<pre class="language-java"><code class="lang-java"><strong>engine.injectMessage(MessageB.create(), true);
</strong></code></pre>

The above is equivalent to the following which injects the messages explicitly specifying the message priority to be 0.

```java
engine.injectMessage(MessageB.create(), true, 0);
```

#### Prioritized Injection

Prioritized injection is performed in the same manner as above but using a negative value for the last parameter - the `delay` parameter. The lower the value supplied, the higher the priority.

The following illustrates how to perform prioritized injection. In this example, the message injected with priority -2 will be dispatched before the message injected with priority -1.

{% hint style="danger" %}
Priorities less than -1000 are reserved for platform use.
{% endhint %}

```java
@EventHandler
public void onMessageA(MessageA message) {
  engine.injectMessage(MessageB.create(), true, -1);
  engine.injectMessage(MessageC.create(), true, -2);
}

@EventHandler
public void onMessageB(MessageB message) {
}

@EventHandler
public void onMessageC(MessageC message) {
   // this handler will get invoked before onMessageB() even though MessageB was injected before Message
}
```

{% hint style="warning" %}
In the above example, the messages will be predictably dispatched in the order listed above because the injection itself is being done from a message handler i.e. by the engine's event multiplexer dispatch thread causing the dispatch to ocuur only after the thread has returned from the message handler meaning that both injected messages are in the multiplexer's queue before the message dispatch starts. If the inject thread is different from the event multiplexer thread, then this behavior is not guaranteed since the multiplexer thread may concurrently dispatch the first enqueed message while the second one is being enqueued.
{% endhint %}

### Injection with Delayed Dispatch <a href="#schedulingandinjectionofmessages-injectionfromanevent-messagehandler" id="schedulingandinjectionofmessages-injectionfromanevent-messagehandler"></a>

Injection with delayed dispatch is performed in the same manner as above but using a positive value for the last parameter - the `delay` parameter. If the `delay` parameter is > 0, then the value is interpreted as the dispatch delay, in milliseconds.

The following illustrates how to perform injection with delayed dispatch. In this example, the injected message will be dispatched to its handler 1 second after injection.

```java
@EventHandler
public void onMessageA(MessageA message) {
  engine.injectMessage(MessageB.create(), true, 1000);
}

@EventHandler
public void onMessageB(MessageB message) {
   // this handler will be invoked 1 second after the injection was done in onMessageA()
}
```

## Injection, Transactions and Processing Notification <a href="#schedulingandinjectionofmessages-delayedorpriorityinjection" id="schedulingandinjectionofmessages-delayedorpriorityinjection"></a>

A injected messages either starts a new transaction or can be slotted into an ongoing transaction. The injector does not have any control on which transaction the injected message eventually ends up in. However, the injector can request to be notified when the processing of the injected messages is complete and the transaction in which the injected message was processed was completed. It does so by supplying an instance of [IEventAcknowledger](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/event/IEventAcknowledger.html) to `injectMessage()`. The supplied instance of the event acknowledger will be invoked when the transaction, containing the injected message, is committed.

The following illustrates how to perform an injection with a request to be notified on completion of the transaction that absorbs the injected message.

```java
engine.injectMessage(MessageB.create(), 
                     true, 
                     new IEventAcknowledger() {
                        @Override
                        public void ack() {
                           // this method will be invoked when the transaction, containing the injected message, completes its commit
                        }
                     });
```

## Blocking vs. Non-Blocking Injection <a href="#schedulingandinjectionofmessages-blockingvs.non-blockingconsiderations" id="schedulingandinjectionofmessages-blockingvs.non-blockingconsiderations"></a>

The `true` parameter in the above examples is a parameter indicating whether the injection should be done in a blocking or non-blocking manner. Blocking injection is injection in which the injector thread blocks in case its queue in the event multiplexer is full. With non-blocking injection, the injector thread never blocks - the queue continues to grow. Injections done by the event multiplexer dispatch thread e.g, injections done from within an event/message handler will never block.

{% hint style="warning" %}
Care must be taken when considering whether to use using blocking or non-blocking injection. If the injecting thread is injecting to an engine multiplexer that may itself block on a resource held by the thread trying to inject, it can cause a deadlock. Conversely, using non-blocking dispatch can result in excessive memory growth, increased latency, and fairness issues. Therefore, if the injecting thread is drawing events from an external source, blocking dispatch is generally the right choice, but if injection is being performed from a message handler, non-blocking should be used.
{% endhint %}

## Injected Message Ownership <a href="#schedulingandinjectionofmessages-messagepoolingconsiderations" id="schedulingandinjectionofmessages-messagepoolingconsiderations"></a>

Message injection transfer ownership of the message to the engine until the processing of the message is complete i.e. until the event acknowledger supplied by the injector in invoked. An injected message must not be written to or read from during this period of time. Once the processing is complete, the engine will `dispose()` its reference to the message (passed to it by the inject call). Therefore, if the user wishes to hold onto the message reference to perform read /write operations after the processing of the injected message is complete, then it would need to acquire a reference to the message via the message's `acquire()` method.

## HA Considerations <a href="#schedulingandinjectionofmessages-behaviorwhennotinhaactiverole" id="schedulingandinjectionofmessages-behaviorwhennotinhaactiverole"></a>

In Event Sourced microservices, messages will only be injected into the event multiplexer of engines of `Primary` cluster instances in the `Started` state. Calls to inject messages on backup instances are ignored since, for Event Sourced engines, the injected message will be injected into and replicated from the primary and, for State Replication engines, all message processing is done only on the primary. Calls made while an engine is replaying from a transaction log (i.e,. engine state is `Starting`) are similarly ignored as those calls would interfere with the stream being replayed. An application that injects messages from an external source may call `AepEngine.waitForMessagingToStart()` to avoid an injected message being discarded while an engine is transitioning to a started, primary role.

{% hint style="warning" %}
It is important to note that message injection is effectively a BestEffort operation because injections of messages that are in the event multiplexer queue at the time of failure will be lost. Reliable injection can be achieved via Message Scheduling instead of Message Injection though fault-tolerant scheduling incurs slightly higher overhead.
{% endhint %}


# Scheduling Messages

{% hint style="info" %}
**Documentation Status**: This page is a placeholder and will be completed in a future update.
{% endhint %}

## Overview

This page will cover how to schedule messages for future or periodic execution in your Talon microservice.

## Topics to be Covered

* Scheduling one-time future messages
* Scheduling recurring/periodic messages
* Canceling scheduled messages
* Best practices for scheduled message patterns

## Related Topics

* [Injecting Messages](/talon/developing-applications/authoring-user-code/message-injection/injecting-messages) - Programmatic message injection
* [Initializing the Microservice](/talon/developing-applications/authoring-user-code/lifecycle/initializing-the-microservice) - First and initial messages


# Command and Control

This section covers how to implement administrative command handlers that allow runtime control and inspection of your microservice.

## Overview

Talon provides a command-and-control mechanism that allows administrators to send commands to running microservices for:

* **Runtime Configuration** - Adjust behavior without restart
* **Diagnostics** - Query internal state and statistics
* **Operations** - Trigger administrative actions

Commands are implemented using annotated methods (`@Command`) and can be invoked via administrative tools or programmatically.

## Command Characteristics

* **Synchronous Execution** - Commands execute immediately and return results
* **Outside Transaction Scope** - Commands don't participate in message transactions
* **Administrative Context** - Designed for operations, not business logic

## Topics

* [**Implementing Command Handlers**](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Create custom administrative commands using `@Command` annotations

## Related Topics

* [Admin Tool](/talon/operating-applications/administration/admin-tool) - Command-line tool for invoking commands
* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Remote command invocation via messaging


# Implementing Command Handlers

{% hint style="info" %}
**Since 3.4**
{% endhint %}

## Overview

A Talon XVM has the ability to discover annotated 'Command' methods provided by an application. Such commands can be invoked remotely by tools in an out of band fashion with an application's message driven event handlers to perform administrative functions. The command framework is designed to work in the context of command line tools, guis or for administrative applications. A command method is identified by annotating it with a @[Command](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Command.html) annotation, and its parameters are exposed by annotating its parameters with either @[Option](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Option.html) or @[Argument](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Argument.html) annotations which allows commands to be invoked by passing the arguments in a command line format.

```java
@Command(name = "resetOrderStats",
         description="Resets the number of orders processed")
public String resetOrderStats(@Option(shortForm = 'v', longForm="verbose", defaultValue="false")
                         boolean verbose,
                         @Argument(name = "newOrderReceivedCount"), position=1)
                         long newNumOrdersReceived,
                         @Argument(name = "newOrderProcessingCount", position=2)
                         long newNumOrdersProcessed) { ... }
```

{% hint style="warning" %}
**Threading Constraint**: Because command handlers are not executed on the microservice's business logic thread **they are not allowed to touch microservice state.** In general applications that intend to pass values back to command handlers should do so through member variables that are not part of the applications HA state that are declare as volatile. Alternatively, a command handler may [inject a message](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/aep/AepEngine.html#injectMessage\(com.neeve.rog.IRogMessage,%20boolean,%20int\)) into the application to perform logic that works with microservice state, but this is considered advanced usage as doing so is not a fault tolerant and blurs the lines between application and administrative traffic.
{% endhint %}

## Annotating Command methods

At its simplest, a method that is composed of primitive arguments needs only to be annotated with a Command annotation to expose it as a command that can be invoked remotely.

**Command Example:**

```java
@App
public void MyApp {
  @AppStat(name = "Orders Processed")
  private volatile long numOrdersProcessed = 0;

  /**
   * Resets the number of orders processed
   *
   * @param newNumOrdersProcessed The new number of orders processed
   * @return The previous number of orders
   */
  @Command
  public int resetOrdersProcessed(long newNumOrdersProcessed,
                                  boolean verbose) {
    long prevNumOrdersProcessed = numOrdersProcessed;
    numOrdersProcessed = numOrdersProcessed;
    String result = "Reset Orders Processed: " + prevNumOrdersProcessed + "->" + numOrdersProcessed;

    if(verbose) {
      System.out.println(result);
    }

    return result;
  }

  /**
   * Handles a new order.
   */
  @EventHandler
  public final void onOrder(NewOrderMessage newOrder) {
    numOrdersReceived++;

    // ... do some processing

    numOrdersProcessed++
  }
}
```

The above method can then be invoked via the [Admin Tool](/talon/operating-applications/administration/admin-tool) in interactive mode:

```bash
invoke MyApp MyXVM resetOrdersProcessed 0 true
```

... the command name is identified by the name of the method and each parameter in the signature is treated as an argument to the command.

### @Command

Above we gave a simple example of using the [Command](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Command.html) annotation to identify a command. The command can be further described by using the following command annotation elements:

| Parameter        | Type       | Description                                                                                                                                                                                                                 | Default |
| ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| **aliases**      | String \[] | Returns the aliases used to invoke this command. This can be useful in cases where a command's name has change and there is a need to support backwards compatibliity.                                                      | none    |
| **description**  | String     | A description of the command. This is used by tools to expose usage information for the command. It is a good idea to provide a description on commands as it provides tools with the ability to provide usage information. | none    |
| **name**         | String     | Returns the name of the command. If this parameter is omitted on an annotated Method then the name will default to the method name.                                                                                         | none    |
| **parseOptions** | boolean    | Whether or not the command parser should attempt to parse options or if this command is only comprised of arguments. This can be useful to avoid needing to escape argument values that may start with a '-' character.     | true    |

### @Argument

The @[Argument](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Argument.html) annotation can be applied to method parameters to describe and constrain arguments.

| Parameter        | Type       | Description                                                                                                                                       | Default |
| ---------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| **name**         | String     | A short name for the argument. This is used to identify the argument when displaying command line usage.                                          | none    |
| **description**  | String     | A brief description desribing what should be supplied for this argument. This is used to describe the argument when displaying command line usage | none    |
| **defaultValue** | String     | The argument's default value (if not a required argument. This value is converted from the String supplied to the argument's type.                | null    |
| **required**     | boolean    | Whether or not the argument is required. It is illegal to position a required argument after an optional one ... optional arguments.              | true    |
| **position**     | int        | The position of the argument on the command line. The 1 based position of the argument for command line invocation.                               | none    |
| **validOptions** | String \[] | Indicates the set of permissible values for the argument.                                                                                         | NULL    |

Below is an example of how an Argument annotation may be used:

```java
@Command(name = "resetOrderStats",
         description="Resets the number of orders processed")
public int resetOrdersProcessed(@Option(shortForm = 'v', longForm="verbose", defaultValue="false")
                                boolean verbose,
                                @Argument(name = "newOrderProcessingCount", position=1)
                                long newNumOrdersProcessed)
```

### @Option

The @[Option](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/cli/annotations/Command.html) annotation can be applied to method parameters. An Option differs from an argument in that it is invoked using a command line switch. Options are useful in cases where a command method signature needs to be changed as scripts written against the old command signature will continue to function.

| Parameter        | Type       | Description                                                                                                                                                  | Default |
| ---------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------- |
| **shortForm**    | char       | The short (switch e.g. -o) form for the option.                                                                                                              | none    |
| **longForm**     | String     | The long form for the argument (e.g. --option)).                                                                                                             | none    |
| **description**  | String     | A brief description of the Option.                                                                                                                           |         |
| **defaultValue** | String     | The option's default value. When the option is not specified the option will be set to this value.                                                           | null    |
| **required**     | boolean    | Whether or not the option is required. If the option is not specified but there is a default value, then the default will be used, otherwise it is an error. | true    |
| **validOptions** | String \[] | Indicates the set of permissible values for the option.                                                                                                      | NULL    |

Below is an example of how an Argument annotation may be used:

```java
@Command(name = "resetOrderStats",
         description="Resets the number of orders processed")
public int resetOrdersProcessed(@Option(shortForm = 'v', longForm="verbose", defaultValue="false")
                                boolean verbose,
                                @Argument(name = "newOrderProcessingCount", position=1)
                                long newNumOrdersProcessed)
```

## Supported Types for Options, Arguments And Return values:

The following types are returned when describing the command usage. In Json they are represented as the name from the enumeration below. The expectation is that tools will use this options in forms.

**Valid Arguments and Options types:**

* boolean or Boolean
* byte or Byte
* char or Char
* short or Short
* int or Integer
* long or Long
* float or Float
* double or Double
* date or Date
* String
* Enumerations (converted to/from a String when sent over the wire).

**Valid return types**

All of the support argument types above and void.

## Registering Command Handlers with the runtime.

### @AppCommandHandlerContainersAccessor

Any @Command annotated method in the main application class will be discovered by a Talon XVM. If additional classes in your application contain @Command methods, they can be exposed to the Talon XVM using the @AppCommandHandlerContainersAccessor annotation which should add all objects that should be introspected for command handlers.

**Example:**

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public static class MyApp {
  MyOtherClass someOtherClass = new MyOtherClass();

  @AppCommandHandlerContainersAccessor
  public void getCommandHandlers(Set<Object> containers) {
    containers.add(someOtherClass );
  }
}

class MyOtherClass {
  volatile int count = 0;
  MyOtherClass() {
  }

  @Command(name = "getCount",
           description="Returns a counter value")
  public int getCount() {
    return count;
  }
}
```

### Configuration Discovery in Hornet

For Topic Oriented Applications, any @Managed object will be introspected for @Command methods. See [ManagedObjectLocator](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/managed/ManagedObjectLocator.html). The [DefaultManagedObjectLocator](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/managed/DefaultManagedObjectLocator.html) for Hornet calls [TopicOrientedApplication.addAppCommandHandlerContainers(Set)](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/toa/TopicOrientedApplication.html#addAppCommandHandlerContainers\(java.util.Set\)), so unless your application provides its own managed object locator, additional configured containers can be added by overriding addConfiguredContainers():

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public static class MyApp {
  MyOtherClass someOtherClass = new MyOtherClass();

  @Override
  public void addAppCommandHandlerContainers(Set<Object> containers) {
    containers.add(someOtherClass );
  }
}

class MyOtherClass {
  volatile int count = 0;
  MyOtherClass() {
  }

  @Command(name = "getCount",

           description="Returns a counter value")
  public int getCount() {
    return count;

  }
}
```

## Invoking Annotated Commands

Annotated command handlers can be discovered and invoked by deployment and administrative tools via admin connections made to the XVM in which the application is running.

## Examples / Appendix

### A Fully Annotated Command

The below command handler exercises all of the annotation features:

```java
@Command(name = "testCommand", aliases = { "TestCommandAlias" }, description = "A command that exercises all of the invocation APIs")
public String testCommand(@Option(shortForm = 'b', longForm = "byteOption", required = true, description = "A byte Option", validOptions = { "0,", "1" }, defaultValue = "0") byte aByteOption,
                          @Option(shortForm = 'c', longForm = "charOption", required = true, description = "A char Option", validOptions = { "a,", "b", "c" }, defaultValue = "b") char aCharOption,
                          @Option(shortForm = 'n', longForm = "shortOption", required = true, description = "A short Option", validOptions = { "399,", "300" }, defaultValue = "399") short aShortOption,
                          @Option(shortForm = 'i', longForm = "intOption", required = true, description = "An int Option", validOptions = { "20000,", "200000" }, defaultValue = "200000") int aIntOption,
                          @Option(shortForm = 'l', longForm = "longOption", required = true, description = "A long Option", validOptions = { "100000000,", "2000000000" }, defaultValue = "100000000") long aLongOption,
                          @Option(shortForm = 'f', longForm = "floatOption", required = true, description = "A float Option", validOptions = { "0.0,", "-2.0" }, defaultValue = "0.0") float aFloatOption,
                          @Option(shortForm = 'd', longForm = "doubleOption", required = true, description = "A double Option", validOptions = { "0.0,", "1.2" }, defaultValue = "1.2") double aDoubleOption,
                          @Option(shortForm = 's', longForm = "stringOption", required = true, description = "A String Option", validOptions = { "Foo,", "Bar" }, defaultValue = "Foo") String aStringOption,
                          @Option(shortForm = 'm', longForm = "currencyOption", required = true, description = "A Currency Option", validOptions = { "USD,", "JPY" }, defaultValue = "JPY") Currency aCurrencyOption,
                          @Option(shortForm = 'e', longForm = "enumOption", required = true, description = "An Enum Option", defaultValue = "PrettyPrint") JsonPrettyPrintStyle aEnumOption,
                          @Argument(position = 1, required = true, name = "aByteArgument", description = "A byte Arguement", validOptions = { "0,", "1" }, defaultValue = "0") byte aByteArgument,
                          @Argument(position = 2, required = true, name = "aCharArgument", description = "A char Arguement", validOptions = { "a,", "b", "c" }, defaultValue = "a") char aCharArgument,
                          @Argument(position = 3, required = true, name = "aShortArgument", description = "A short Arguement", validOptions = { "399,", "300" }, defaultValue = "300") short aShortArgument,
                          @Argument(position = 4, required = true, name = "aIntArgument", description = "An int Arguement", validOptions = { "20000,", "200000" }, defaultValue = "200000") int aIntArgument,
                          @Argument(position = 5, required = true, name = "aLongArgument", description = "A long Arguement", validOptions = { "100000000,", "2000000000" }, defaultValue = "100000000") long aLongArgument,
                          @Argument(position = 6, required = true, name = "aFloatArgument", description = "A float Arguement", validOptions = { "0.0,", "-2.01" }, defaultValue = "-2.01") float aFloatArgument,
                          @Argument(position = 7, required = true, name = "aDoubleArgument", description = "A double Arguement", validOptions = { "0.0,", "1.2" }, defaultValue = "0.0") double aDoubleArgument,
                          @Argument(position = 8, required = true, name = "aStringArgument", description = "A String Arguement", validOptions = { "Foo,", "Bar" }, defaultValue = "Foo") String aStringArgument,
                          @Argument(position = 9, required = true, name = "aCurrencyArgument", description = "A Currency Arguement", validOptions = { "USD,", "JPY" }, defaultValue = "USD") Currency aCurrencyArgument,
                          @Argument(position = 10, required = true, name = "aEnumArgument", description = "An Enum Arguement", defaultValue = "Minimal") JsonPrettyPrintStyle aEnumArgument) throws Exception {
    return "Received Arguments: " + Arrays.asList(new Object[] {
                                                                aByteOption,
                                                                aCharOption,
                                                                aShortOption,
                                                                aIntOption,
                                                                aLongOption,
                                                                aFloatOption,
                                                                aDoubleOption,
                                                                aStringOption,
                                                                aCurrencyOption,
                                                                aEnumOption,
                                                                aByteArgument,
                                                                aCharArgument,
                                                                aShortArgument,
                                                                aIntArgument,
                                                                aLongArgument,
                                                                aFloatArgument,
                                                                aDoubleArgument,
                                                                aStringArgument,
                                                                aCurrencyArgument,
                                                                aEnumArgument }).toString();
}
```

#### Usage as Printed By SrvMonUtil

```
testCommand
A command that exercises all of the invocation APIs
 Usage:
  testCommand -b -c -n -i -l -f -d -s -m -e <aByteArgument> <aCharArgument> <aShortArgument> <aIntArgument> <aLongArgument> <aFloatArgument> <aDoubleArgument> <aStringArgument> <aCurrencyArgument> <aEnumArgument>
       <-b|--byteOption> <1|0,> Tests a byte Option default='0'
       <-c|--charOption> <a,|b|c> Tests a char Option default='b'
       <-n|--shortOption> <300|399,> Tests a short Option default='399'
       <-i|--intOption> <200000|20000,> Tests a int Option default='200000'
       <-l|--longOption> <100000000,|2000000000> Tests a long Option
           default='100000000'
       <-f|--floatOption> <0.0,|-2.0> Tests a float Option default='0.0'
       <-d|--doubleOption> <0.0,|1.2> Tests a double Option default='1.2'
       <-s|--stringOption> <Bar|Foo,> Tests a String Option default='Foo'
       <-m|--currencyOption> <JPY|USD,> Tests a Currency Option
           default='JPY'
       <-e|--enumOption> Tests a Enum Option default='PrettyPrint'
       [aByteArgument: <1|0,> Tests a byte Arguement default='0']
       [aCharArgument: <a,|b|c> Tests a char Arguement default='a']
       [aShortArgument: <300|399,> Tests a short Arguement default='300']
       [aIntArgument: <200000|20000,> Tests a int Arguement
           default='200000']
       [aLongArgument: <100000000,|2000000000> Tests a long Arguement
           default='100000000']
       [aFloatArgument: <0.0,|-2.01> Tests a float Arguement
           default='-2.01']
       [aDoubleArgument: <0.0,|1.2> Tests a double Arguement default='0.0']
       [aStringArgument: <Bar|Foo,> Tests a String Arguement default='Foo']
       [aCurrencyArgument: <JPY|USD,> Tests a Currency Arguement
           default='USD']
       [aEnumArgument: Tests a Enum Arguement default='Minimal']
```

#### Json Command Description:

```json
{
  "name": "testCommand",
  "aliases": [
    "TestCommandAlias"
  ],
  "description": "A command that exercises all of the invocation APIs",
  "additionalArguments": false,
  "arguments": [
    {
      "position": 1,
      "required": true,
      "type": "BYTE",
      "name": "aByteArgument",
      "defaultValue": "0",
      "validValues": [
        "1",
        "0,"
      ],
      "description": "Tests a byte Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 2,
      "required": true,
      "type": "CHAR",
      "name": "aCharArgument",
      "defaultValue": "a",
      "validValues": [
        "a,",
        "b",
        "c"
      ],
      "description": "Tests a char Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 3,
      "required": true,
      "type": "SHORT",
      "name": "aShortArgument",
      "defaultValue": "300",
      "validValues": [
        "300",
        "399,"
      ],
      "description": "Tests a short Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 4,
      "required": true,
      "type": "INT",
      "name": "aIntArgument",
      "defaultValue": "200000",
      "validValues": [
        "200000",
        "20000,"
      ],
      "description": "Tests a int Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 5,
      "required": true,
      "type": "LONG",
      "name": "aLongArgument",
      "defaultValue": "100000000",
      "validValues": [
        "100000000,",
        "2000000000"
      ],
      "description": "Tests a long Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 6,
      "required": true,
      "type": "FLOAT",
      "name": "aFloatArgument",
      "defaultValue": "-2.01",
      "validValues": [
        "0.0,",
        "-2.01"
      ],
      "description": "Tests a float Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 7,
      "required": true,
      "type": "DOUBLE",
      "name": "aDoubleArgument",
      "defaultValue": "0.0",
      "validValues": [
        "0.0,",
        "1.2"
      ],
      "description": "Tests a double Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 8,
      "required": true,
      "type": "STRING",
      "name": "aStringArgument",
      "defaultValue": "Foo",
      "validValues": [
        "Bar",
        "Foo,"
      ],
      "description": "Tests a String Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 9,
      "required": true,
      "type": "STRING",
      "name": "aCurrencyArgument",
      "defaultValue": "USD",
      "validValues": [
        "JPY",
        "USD,"
      ],
      "description": "Tests a Currency Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "position": 10,
      "required": true,
      "type": "STRING",
      "name": "aEnumArgument",
      "defaultValue": "Minimal",
      "description": "Tests a Enum Arguement",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    }
  ],
  "options": [
    {
      "shortForm": "b",
      "required": true,
      "type": "BYTE",
      "longForm": "byteOption",
      "defaultValue": "0",
      "validValues": [
        "1",
        "0,"
      ],
      "description": "Tests a byte Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "c",
      "required": true,
      "type": "CHAR",
      "longForm": "charOption",
      "defaultValue": "b",
      "validValues": [
        "a,",
        "b",
        "c"
      ],
      "description": "Tests a char Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "n",
      "required": true,
      "type": "SHORT",
      "longForm": "shortOption",
      "defaultValue": "399",
      "validValues": [
        "300",
        "399,"
      ],
      "description": "Tests a short Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "i",
      "required": true,
      "type": "INT",
      "longForm": "intOption",
      "defaultValue": "200000",
      "validValues": [
        "200000",
        "20000,"
      ],
      "description": "Tests a int Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "l",
      "required": true,
      "type": "LONG",
      "longForm": "longOption",
      "defaultValue": "100000000",
      "validValues": [
        "100000000,",
        "2000000000"
      ],
      "description": "Tests a long Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "f",
      "required": true,
      "type": "FLOAT",
      "longForm": "floatOption",
      "defaultValue": "0.0",
      "validValues": [
        "0.0,",
        "-2.0"
      ],
      "description": "Tests a float Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "d",
      "required": true,
      "type": "DOUBLE",
      "longForm": "doubleOption",
      "defaultValue": "1.2",
      "validValues": [
        "0.0,",
        "1.2"
      ],
      "description": "Tests a double Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "s",
      "required": true,
      "type": "STRING",
      "longForm": "stringOption",
      "defaultValue": "Foo",
      "validValues": [
        "Bar",
        "Foo,"
      ],
      "description": "Tests a String Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "m",
      "required": true,
      "type": "STRING",
      "longForm": "currencyOption",
      "defaultValue": "JPY",
      "validValues": [
        "JPY",
        "USD,"
      ],
      "description": "Tests a Currency Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    },
    {
      "shortForm": "e",
      "required": true,
      "type": "STRING",
      "longForm": "enumOption",
      "defaultValue": "PrettyPrint",
      "description": "Tests a Enum Option",
      "_xFieldBitmask_": [
        0
      ],
      "xRogType": 0
    }
  ],
  "returnType": "VOID",
  "_xFieldBitmask_": [
    0
  ],
  "xRogType": 0
}
```

## Related Topics

* [Admin Tool](/talon/operating-applications/administration/admin-tool) - Command-line administrative interface
* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Remote administration via messaging
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Custom statistics

## Next Steps

1. Define command handlers for your application
2. Annotate methods with @Command, @Argument, and @Option
3. Register command handler containers if needed
4. Test commands using the Admin Tool or administrative tools
5. Document available commands for operators


# Configuration

This section covers how to inject configuration values into your microservice application code.

## Overview

Talon provides annotation-driven configuration injection that allows you to initialize fields and methods with configuration values from the XRuntime environment. This provides a declarative alternative to programmatic access via the `XRuntime.getValue()` API.

## Topics

* [**Injecting Configuration**](/talon/developing-applications/authoring-user-code/configuration/injecting-configuration) - Using @Configured annotation for configuration injection

## Related Topics

* [Configuration Model](/talon/concepts-and-architecture/microservice-architecture/configuration-model) - Understanding Talon's configuration architecture
* [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Configuration injection in the microservice lifecycle


# Injecting Configuration

{% hint style="info" %}
**Since 3.2**
{% endhint %}

## Overview

The platform provides the ability to initialize @Configured annotated fields and methods with configuration values from the XRuntime. This provides a declarative alternative to using the `XRuntime.getValue()` API programmatically.

Annotated fields and methods are discovered automatically for the main Talon application class. Additional classes can be exposed using the @AppConfiguredAccessor or @AppIntrospectionPoints annotations.

## The @Configured Annotation

The @Configured annotation is used to annotate application fields and methods to allow them to be discovered by the Talon XVM. The Talon XVM will populate these fields and methods with settings from the Talon configuration.

### Annotation Elements

| Element          | Type              | Description                           | Default  |
| ---------------- | ----------------- | ------------------------------------- | -------- |
| **property**     | String (required) | The configuration property name       | -        |
| **required**     | boolean           | Whether the property is required      | false    |
| **defaultValue** | String            | The default value if property not set | `<null>` |
| **description**  | String            | A description of the property         | `<null>` |

### Supported Types

Configuration values can be of the following types:

* boolean
* byte
* short
* int
* long
* float
* double
* char
* String or XString
* enumerations

## Using @Configured on Fields

You can annotate fields directly for configuration injection:

```java
import com.neeve.cli.annotations.Configured;

@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {

  @Configured(property = "simulator.ems.orderPreallocateCount", defaultValue = "1048576")
  private int orderPreallocateCount;

  @Configured(property = "myapp.maxOrderSize", required = true)
  private int maxOrderSize;
}
```

## Using @Configured on Setter Methods

You can also annotate setter methods for configuration injection:

```java
import com.neeve.cli.annotations.Configured;

@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
  private int orderPreallocateCount;

  @Configured(property = "simulator.ems.orderPreallocateCount", defaultValue = "1048576")
  void setOrderPreallocateCount(int orderPreallocateCount) {
    this.orderPreallocateCount = orderPreallocateCount;
  }
}
```

## Configuration Discovery

### Main Application Class

Any @Configured annotated field or method in the main application class will be discovered automatically by the Talon XVM. No additional setup is required.

### Additional Classes with @AppConfiguredAccessor

If additional classes in your application contain configured fields or methods, they can be exposed to the XVM using the @AppConfiguredAccessor annotation:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
    MyOtherClass someOtherClass = new MyOtherClass();

    @AppConfiguredAccessor
    public void getConfiguredContainers(Set<Object> containers) {
        containers.add(someOtherClass);
    }
}

private class MyOtherClass {
    @Configured(property = "simulator.ems.orderPreallocateCount", defaultValue = "1048576")
    private int orderPreallocateCount;

    MyOtherClass() {
    }
}
```

### Additional Classes with @AppIntrospectionPoints

The @AppIntrospectionPoints annotation exposes a collection of application objects to introspect for any type of annotation supported by Talon, including @Configured. This is a broader annotation that can be used instead of the more narrowly scoped @AppConfiguredAccessor:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public class MyApp {
    MyOtherClass someOtherClass = new MyOtherClass();

    @AppIntrospectionPoints
    public void getApplicationObjects(Set<Object> objects) {
        objects.add(someOtherClass);
    }
}

private class MyOtherClass {
    @Configured(property = "simulator.ems.orderPreallocateCount", defaultValue = "1048576")
    private int orderPreallocateCount;

    MyOtherClass() {
    }
}
```

{% hint style="info" %}
**Choosing Between Annotations**

Use @AppConfiguredAccessor for applications that have a large number of objects to reduce the number of objects that the XVM needs to scan for annotations. Use @AppIntrospectionPoints when you want to expose objects for multiple annotation types (@Configured, @EventHandler, @AppStat, @Command, etc.) to reduce boilerplate code.
{% endhint %}

## Limitations

In order to avoid potential race conditions with static fields and preserve the semantics of the "final" keyword, the @Configured annotation does not support injecting properties into fields that are declared static or final.

If you attempt to annotate a static or final field with @Configured, the framework will throw a CliException at initialization time.

## Programmatic Access to Configuration

Configuration settings remain accessible via the `XRuntime.getValue()` API even when using annotation-driven configuration:

```java
String value = XRuntime.getValue("my.property.name", "defaultValue");
```

## Lifecycle Integration

Configuration injection occurs during the microservice lifecycle after the engine descriptor is prepared. The sequence is:

1. Talon XVM loads the microservice main class
2. XVM injects the application loader
3. XVM prepares the engine descriptor
4. **XVM calls @AppConfiguredAccessor (if present)**
5. **XVM performs configuration injection on discovered objects**
6. XVM continues with remainder of lifecycle (command handlers, event handlers, etc.)

See [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) for complete lifecycle details.

## See Also

* [Lifecycle](/talon/concepts-and-architecture/microservice-operation/lifecycle) - Microservice lifecycle and configuration injection timing
* [Configuration Model](/talon/concepts-and-architecture/microservice-architecture/configuration-model) - Understanding Talon's configuration architecture
* [Implementing Lifecycle Methods](/talon/developing-applications/authoring-user-code/lifecycle/implementing-lifecycle-methods) - Other lifecycle injection points


# Monitoring

This section covers how to expose custom statistics and telemetry from your microservice for monitoring and observability.

## Overview

Talon automatically collects extensive statistics about message processing, transactions, and system resources. You can augment this with application-specific metrics to monitor business logic and operational characteristics.

Custom statistics are exposed through heartbeat logs and can be queried in real-time or analyzed offline.

## Statistic Types

Talon supports four types of custom statistics:

* **Gauges** - Current value measurements (e.g., queue depth, cache size)
* **Counters** - Monotonically increasing counts (e.g., orders processed, errors)
* **Series** - Collections of measurements (e.g., order sizes, processing times)
* **Latencies** - Timing measurements with percentile tracking

## Topics

* [**Exposing Application Stats**](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Define custom statistics using `@AppStat` annotations

## Related Topics

* [Operating Model](/talon/concepts-and-architecture/operating-model) - Conceptual overview of monitoring and operations
* [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats) - Server-level statistics
* [Engine Stats](/talon/operating-applications/monitoring/engine-statistics) - Engine-level metrics
* [Configuring Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Configure statistics collection


# Exposing Application Statistics

{% hint style="info" %}
**Since 3.1**
{% endhint %}

## Overview

The platform provides the ability for users to define their own application specific stats. These user defined app stats can be registered with the AepEngine, which allows them to be traced along with [Engine Stats](/talon/operating-applications/monitoring/engine-statistics), and be included in XVM heartbeats. Applications can programmatically register stats with the AepEngine, or when running in a Talon XVM to be discovered via annotations.

This article describes the usage of the following types of statistics that applications can expose:

| Stat Type     | Description                                                                                                                                                                                  |
| ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Gauge**     | A Gauge samples and reports a value on each stats collection interval. Gauges can be exposed simply by annotating a field or method of interest.                                             |
| **Counter**   | A Counter stat captures a monotonically increasing value, and can be used to derive rates based on deltas between intervals.                                                                 |
| **Series**    | A Series stat allows recording of a series of data points upon which histographical computations can be reported.                                                                            |
| **Latencies** | A special case of a Series stat used to collected timing related data points. Latencies are used extensively within Core X to provide visibility into transaction processing pipeline times. |

## Gauges

A Gauge captures an instantaneous value at the time of a statistic collection. Gauges can be of the following types:

* `boolean`
* `byte`
* `short`
* `int`
* `long`
* `float`
* `double`
* `char`
* `String` or `XString`

{% hint style="warning" %}
**Important considerations regarding gauge collection**

Gauge values are collected on a stats collection thread(s) separate from the business logic thread. Consequently:

* Gauge field values must be declared as volatile to ensure changes to them are visible to collection threads.

Additionally, for method gauges:

* Be sure that the computation cost is not so high that it skews statistics collection. Consider using a background thread for computing gauge values that are computationally expensive.
* It is possible that more than one stats collection thread will be collecting and reporting on stats concurrently, so method gauges should be threadsafe.
  {% endhint %}

### Field Gauges

When running in a Talon XVM, it is possible to annotate a field as a gauge:

```java
import com.neeve.stats.*;

public void MyApp() {

  @AppStat(name="Last Order ID Processed")
  private volatile int lastOrderNumber = -1;

  @EventHandler
  public void onNewOrder(NewOrderMessage message) {
     lastOrderNumber = message.getOrderId();
  }
}
```

### Method Gauges

When running in a Talon XVM, it is possible to annotate a method as a gauge accessor:

```java
import com.neeve.stats.*;

public void MyApp() {
  private volatile int numInvalidOrders;

  @EventHandler
  public void onNewOrder(NewOrderMessage message) {
    if(message.getQuantity() < 0) {
      numInvalidOrders.increment();
    }
  }

  @AppStat(name = "Invalid Order Flag")
  public boolean getHasOrderErrors() {
    return numInvalidOrders > 0;
  }
}
```

{% hint style="warning" %}
Method accessor gauges for primitive type are not Zero Garbage. This is because the platform invokes getHasOrderErrors via reflection which generates autoboxing garbage. A better approach, if your application is sensitive to garbage, is to use a Gauge subclass which can directly return the primitive type.
{% endhint %}

### Gauge Subclass Field

You can subclass one of the XXXGauge implementations to avoid garbage associated with an annotated method. This is useful if your Gauge needs to be calculated or you are not running in a Talon XVM and you need to programmatically register a Gauge instance with the AepEngine.

```java
import com.neeve.stats.*;

public void MyApp() {
  private volatile int numInvalidOrders;

  @AppStat
  private final Gauge orderErrorsGauge = new BooleanGauge("Invalid Order Flag") {
    public boolean getBooleanValue() {
      return numInvalidOrders > 0;
    }
  };

  @EventHandler
  public void onNewOrder(NewOrderMessage message) {
    if(message.getQuantity() < 0) {
      numInvalidOrders.increment();
    }
  }
}
```

Note that in the above case the 'name' attribute is omitted on the AppStat annotation because it is provided directly when creating the Gauge.

### Gauges on Server Heartbeats

Gauges can be read programmatically on XVM heartbeats:

```java
public class MyStatsListener {

  @EventHandler
  public void onHeartbeat(SrvMonHeartbeatMessage message) {
    for (SrvMonAppStats appStats : message.getAppsStatsEmptyIfNull()) {
      for (SrvMonUserGaugeStat gauge: appStats.getUserStats().getGaugesEmptyIfNull()) {
         System.out.println(gauge.getName() + ": " +
                            SrvMonUtil.getGaugeValue(gauge) +
                            " (" + gauge.getGaugeType() + ")");
      }
    }
  }
}
```

yields:

```
Invalid Order Flag: true
Last Order ID Processed: 10
```

### Gauges in Aep Engine Stats Trace

```
[User Gauge Stats]
...Invalid Orders: 9
...Last Order ID Processed: 10
```

### Threading Considerations for Gauges

Note that in the above examples, gauges fields are declared as volatile. This is because gauge values are collected by the statistics thread that is emitting XVM heartbeats, not the microservice's business logic thread.

## Counters

A `Counter` is useful for recording a monotonically increasing value over time. Sampled periodically, it can be used to derive a rate. For example, a counter could be used to record a number of message received. By sampling it over time, it can be used to create a received message rate.

```java
import com.neeve.stats.IStats.Counter;
import com.neeve.stats.StatsFactory;

@AppStat
private final Counter numInvalidOrders = StatsFactory.createCounterStat("Invalid Orders");

public void MyApp() {

  @EventHandler
  public void onNewOrder(NewOrderMessage message) {
    if(message.getQuantity() < 0) {
      numInvalidOrders.increment();
    }
  }
}
```

If Aep engine stats tracing is enabled, the above stat will be printed along with the rest of engine stats in the format:

`<overallCount> <lastIntervalCount> (<overallRate> <lastIntervalRate>)`:

```
[User Counter Stats]
...Invalid Orders: 9 1 (1.01 1)
```

From the above, we can see that there were 9 invalid orders in the lifetime of the app, 1 invalid order in the last interval, and that the app is receiving a little over 1 invalid order / sec.

User stats are also included in Server Heartbeats, the following code iterates through all user Counter stats and prints them out.

```java
public class MyStatsListener {

  @EventHandler
  public void onHeartbeat(SrvMonHeartbeatMessage message) {
    for (SrvMonAppStats appStats : message.getAppsStatsEmptyIfNull()) {
      for (SrvMonUserCounterStat counter : appStats.getUserStats().getCounters()) {
         System.out.println(counter.getName() + ": " + counter.getCount());
      }
    }
  }
}
```

```
Invalid Orders: 9
```

## Series

Series stats allow capture of a series of datapoints and allow reporting of histographical statistics based on that series.

A common usecase for a Series statistic is collecting Latency timing data. In that, one would like to be able to observe median, min, max 99.99% for message processing times to ensure that SLAs are being met. However, Non-lossy collection and reporting of histographical latency statistics is a challenging problem in low latency systems due to the number of data points that need to be retained, computed and serialized. For example, imagine an application that is recording latency statistics for messages coming in at a rate of 10k/sec. To accurately compute and report percentiles with a collection period of 10 seconds, the application needs to retain at least 100,000 data points per statistic to perform histographical analysis for just one interval! Assuming that the values are double or long values, then one would be looking at \~800Kb per statistic collected. Collecting and computing on such data is hard on processor memory caches and can have a disruptive impact on application processing times. Furthermore, to perform longer term histographical analysis (across multiple collection periods) without losing any data, each set of interval results needs to be stored so that computation can be performed. Persisting such data to disk or emitting it in XVM heartbeats to achieve this is also problematic because it leads to a large volume of data which puts a strain on disk space and bandwidth, or in the case of heartbeats, network bandwidth when emitted over the messaging fabric.

### Loss-less series stats collection

Talon supports the ability to perform loss-less series capture by allowing all collecting latencies timing datapoints to be emitted in heartbeats. Providing that the collection period doesn't exceed the data point capture rate, every datapoint can be emitted in heartbeats (which can be logged to disk or emitted over an SMA channel). However, this approach should be use sparingly as it is quite expensive.

### Histogram (HDR) collection

As an alternative to reporting all captured data points, 3.1 introduces computed histogram reporting based on [HDRHistogram](https://github.com/HdrHistogram/HdrHistogram) which significantly reduces the size of heartbeats by maintaining a running computation of latency statistics. At each collection interval the captured latencies are fed into both a running histogram and an interval histogram.

{% hint style="info" %}
**Running Stats**: These stats allow a monitoring application to connect at any time and get a view into the historical latency statistics (at least since they were last reset).

**Interval Stats**: These stats allow a monitoring application to get an instantaneous view of the statistic over a recent time window.
{% endhint %}

An HDRHistogram compromises on precision of the captured latencies in favor of cheaper computation and storage of results while still maintaining a predictable precision. The documentation on HDR histogram provides details on the level of precision that is achieved. Practically speaking, however, for latency data points in the 100s of microseconds the precision that is guaranteed for collected percentiles is in the order of +/- 1us, which is acceptable for most applications (for tail values, say in the range of 1 minute, the value is guaranteed to be correct within +/- 60ms).

### Creating a Series Stat

```java
import com.neeve.stats.IStats.Series;
import com.neeve.stats.StatsFactory;

@AppStat
private final Series newCustomerAge = StatsFactory.createSeriesStat("New Customer Age");

public void MyApp() {

  @EventHandler
  public void onNewCustomer(NewCustomerCreation message) {
    newCustomerAge.add(message.getQuantity());
  }
}
```

When Aep Engine Statistics are enabled, the statistic would then be traced:

```
[App (myapp) User Stats]
...Series{
......[New Customer Age(sno=8, #points=1, #skipped=0)
.........New Customer Age(interval): [sample=1, min=21 max=21 mean=21 median=21 75%ile=21 90%ile=21 99%ile=21 99.9%ile=21 99.99%ile=21]
.........New Customer Age (running): [sample=8, min=21 max=29 mean=23 median=21 75%ile=22 90%ile=23 99%ile=29 99.9%ile=29 99.99%ile=29]
...}
```

In the above we can see that in the last interval, one new customer registered and their age was 21. Over the last 8 intervals, the average new customer age is 23 with the oldest being 29 and the youngest being 21.

### Series Data in Server Heartbeats

Series data for user stats are exposed in the Server Monitoring Heartbeat using the SrvMonUserSeriesStat object:

```java
public class MyStatsListener {

  @EventHandler
  public void onHeartbeat(SrvMonHeartbeatMessage message) {
    for (SrvMonAppStats appStats : message.getAppsStatsEmptyIfNull()) {
      for (SrvMonUserSeriesStat series: appStats.getUserStats().getSeries()) {
         System.out.println(series.getName() + ": mean: " + series.getIntSeries().getRunningStats().getMean());
      }
    }
  }
}
```

### SrvMonUserSeriesStat

Reports an application defined series statistic.

| Field Name   | Type             | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| ------------ | ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`       | String           | <p>When the XVM is configured to include the capture data points for the statistic, the returned array will include the values collected during this interval. This allows monitoring tools to perform non-lossy calculation of percentiles, providing new data points were skipped due to under sampling or a missed heartbeat.<br><br>The number of valid values in the returned array is dictated by numDataPoints; if the length of the values array is longer than numDataPoints, subsequent values in the array should be ignored.</p> |
| `seriesType` | SrvMonSeriesType | <p>The type of the series data.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br>Currently only Integer Data series are supported. The types BYTE, SHORT, LONG, FLOAT and DOUBLE are reserved for future use. Processors of heartbeats should ensure that they check the data type here for future proofing.<br></p></div>                                                                                                                                                         |
| `intSeries`  | SrvMonIntSeries  | <p>The collected int series data for an INT series.<br><br>This field should only be set when the series type is set to SrvMonSeriesType.INT.</p>                                                                                                                                                                                                                                                                                                                                                                                            |

### SrvMonIntSeries

Latency statistics are reported in a SrvMonIntSeries object.

SrvMonIntSeries reports interval and running histogram data for a series of integer data points. It may also be used to report the captured datapoints, but because reporting the raw data is costly (both in terms of collection and size/bandwidth), the captured values are typically not reported.

SrvMonIntSeries is frequently used to capture measured latency timings, but can also be used to capture any integer data series.

| Field Name           | Type               | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| -------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `dataPoints`         | int\[]             | <p>When the XVM is configured to include the capture data points for the statistic, the returned array will include the values collected during this interval. This allows monitoring tools to perform non-lossy calculation of percentiles, providing new data points were skipped due to under sampling or a missed heartbeat.<br><br>The number of valid values in the returned array is dictated by numDataPoints; if the length of the values array is longer than numDataPoints, subsequent values in the array should be ignored.</p>                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `lastSequenceNumber` | long               | <p>Sequence numbers for collected data points start at 1, a value of 0 indicates that no data points have been collected.<br><br>The Sequence Number always indicates the number or data points that have been collected since the statistic has been created or was last reset.<br>If the statistic is reset then this value will reset to 0.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `numDataPoints`      | int                | <p>Indicates the number of data points collected in this interval. If no data points were collected, numDataPoints will be 0.<br><br>The sequence number of the first value collected in this interval can be determined by subtracting numDataPoints from lastSequenceNumber. This can be used to determine if two consecutive datapoints have skipped data points due to under sampling or a missing heartbeat.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `skippedDataPoints`  | long               | <p>The runtime only holds on to a fixed number of data points for any particular Latency statistic. If the sampling interval is too high, then some datapoints may be skipped. For example, let's say Latency stats are configured to hold on to a sample size of 1000 datapoints. If the number of data points being captured per second is 2000, and the stats collection interval is 1 second, then on each collection, 1000 datapoints will be missed, which will skew results.<br><br>The skipped data points counter thus indicates how many data points have been missed in the reported runningStats. And if the count grows over two successive heartbeats, this indicates that the values the intervalStats don't reflect all the activity since the last interval.<br><br>The skipped data points counter is a running counter: it tracks the total number of data points that have been skipped since the underlying statistic was last reset.</p> |
| `intervalStats`      | SrvMonIntHistogram | <p>Holds computed results for the datapoints captured for this heartbeat (e.g. for the numDataPoints captured).<br><br>This field may not be set if numDataPoints is 0 or if interval computations are not done on the XVM.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `runningStats`       | SrvMonIntHistogram | <p>Holds computed results for the datapoints over the lifetime of this statistic (e.g. since seqNo 1).<br><br>If the underlying statistic is reset then the running stats are also corresponding reset.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |

### SrvMonIntHistogram

Holds calculated statistics of a range of integer datapoints. The values are computed using an [HDRHistogram](https://github.com/HdrHistogram/HdrHistogram).

| Field Name        | Type | Description                                                                                                                                                                                                                                                                                                                                                                                                                               |
| ----------------- | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sampleSize`      | long | The number of datapoints over which results were calculated (possibly 0 if no data points were collected).                                                                                                                                                                                                                                                                                                                                |
| `minimum`         | int  | <p>The minimum value recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                                 |
| `maximum`         | int  | <p>The maximum value recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                                 |
| `mean`            | int  | <p>The mean for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                           |
| `median`          | int  | <p>The median for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                         |
| `pct75`           | int  | <p>The 75th percentile for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                |
| `pct90`           | int  | <p>The 90th percentile for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                |
| `pct99`           | int  | <p>The 99th percentile for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                                |
| `pct999`          | int  | <p>The 99.9th percentile for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                              |
| `pct9999`         | int  | <p>The 99.99th percentile for the values recorded in the sample set.<br><br>The value is not set if the sample size is 0.</p>                                                                                                                                                                                                                                                                                                             |
| `samplesOverMax`  | long | <p>The number of samples that exceeded the maximum recordable value for the histogram.<br><br>When computing latency percentiles using an HDRHistogram, it is possible that a recorded value will exceed the maximum value allowable. In this case, the datapoint is downsampled to the maximum recordable value, which skews the percentile calculations lower. SamplesOverMax allows detection of how frequently this is occurring.</p> |
| `samplesUnderMin` | long | <p>The number of samples captured that were below the recordable value for the histogram.<br><br>When computing latency percentiles using an HDRHistogram, it is possible that a recorded value will be below 0 in cases where clock skew is possible. In such cases, the value will be upsampled to 0, which can skew the histogram results. SamplesUnderMin allows detection of how frequently this is happening.</p>                   |

{% hint style="info" %}
**See also**: [ISrvMonUserSeriesStat](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/server/mon/ISrvMonUserSeriesStat.html), [ISrvMonIntSeries](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/ISrvMonIntSeries.html) and [ISrvMonIntHistogram](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/ISrvMonIntHistogram.html) for details on the values they report.
{% endhint %}

## Latencies

The Latencies stats is an extension of the Series stat. When capturing latency or timing data, it is good practice to use Latencies instead of Series.

```java
import com.neeve.stats.IStats.Latencies;
import com.neeve.stats.StatsFactory;

@AppStat
private final Latencies orderPrepTime = StatsFactory.createLatencyStat("Order Prep Times");

public void MyApp() {

  @EventHandler
  public void onNewOrder(NewOrderMessage message) {
    long receiveTs= UtlTime.now();
    //Do some stuff
    ...
    //Prepped ... capture prep time.
    orderPrepTime.add(UtlTime.now() - receiveTs);
  }
}
```

## User Defined Statistic Discovery

### @AppStat Annotation

The AppStat annotation can be used to annotate user defined statistics in the microservice to allow those statistics to be discovered by a Talon XVM. The Talon XVM will register each statistic it finds with the microservice's AepEngine. AppStat annotations are only introspected once: just after the microservice's AepEngine is injected. If the microservice changes the instance after microservice initialization, the new stat instance won't be discovered by the microservice.

### @AppStatContainersAccessor

Any @AppStat annotated field in the main microservice class will be discovered by the Talon XVM: if additional classes in your microservice contain user defined stats, they can be exposed to the XVM using the AppStatContainerAccessor annotation.

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public static class MyApp {
    MyOtherClass someOtherClass = new MyOtherClass();

    @AppStatContainersAccessor
    public void getStatContainers(Set<Object> containers) {
        containers.add(someOtherClass);
    }
}

private static class MyOtherClass {
    @AppStat
    Counter numHeartbeats = StatsFactory.createCounterStat("Heartbeats Received");
    StatContainer() {
    }
}
```

### AppStat Discovery in Hornet

For Topic Oriented Applications, any @Managed object will be introspected for User Defined stats. See [ManagedObjectLocator](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/managed/ManagedObjectLocator.html). The [DefaultManagedObjectLocator](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/managed/DefaultManagedObjectLocator.html) for Hornet calls [TopicOrientedApplication.addAppStatContainers(Set)](https://build.neeveresearch.com/core/javadoc/LATEST/SNAPSHOT/com/neeve/toa/TopicOrientedApplication.html#addAppStatContainers\(java.util.Set\)), so unless your application provides its own managed object locator, additional user defined stats containers can be added by overriding addAppStatsContainers:

```java
@AppHAPolicy(HAPolicy.EventSourcing)
public static class MyApp extends TopicOrientedApplication {
    MyOtherClass someOtherClass = new MyOtherClass();

    @Override
    public void addAppStatsContainers(Set<Object> containers) {
        containers.add(someOtherClass);
    }
}

private static class MyOtherClass {
    @AppStat
    Counter numHeartbeats = StatsFactory.createCounterStat("Heartbeats Received");
    StatContainer() {
    }
}
```

### Programmatically Registering Stats

When running in a Talon XVM, the XVM registers discovered App Stats with the AepEngine. When *not* running in a Talon XVM, user defined stats may be registered programmatically with the AepEngine by calling the appropriate register method:

| Type                       | Method                                        |
| -------------------------- | --------------------------------------------- |
| Counter                    | `registerCounterStat(IStats.Counter counter)` |
| Gauge                      | `registerGaugeStat(IStats.Gauge gauge)`       |
| <p>Series<br>Latencies</p> | `registerSeriesStat(IStats.Series series)`    |

If not registered with the engine, app stats will not be collected with other engine stats when engine stats are enabled.

{% hint style="warning" %}
Registration of User Defined stats is only supported prior to engine startup.
{% endhint %}

## Related Topics

* [Engine Stats](/talon/operating-applications/monitoring/engine-statistics) - Engine-level statistics
* [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats) - XVM heartbeat configuration

## Next Steps

1. Determine which stat types best fit your application metrics
2. Annotate fields or methods with @AppStat
3. Enable XVM heartbeats to collect statistics
4. Monitor stats via heartbeat handlers or Admin tools
5. Optimize for zero-garbage if needed using Gauge subclasses


# Trace Logging

Learn how to implement trace logging in your Talon microservices.

## Overview

Talon provides a powerful trace logging framework that allows you to emit diagnostic trace from your application code. You can create custom tracers, control trace levels programmatically, and configure output destinations for your application's trace messages.

## Topics

* [**Logging Trace**](/talon/developing-applications/authoring-user-code/trace-logging/logging-trace) - Create tracers and emit trace from your application code

## Related Topics

* [Trace Logging Configuration](/talon/operating-applications/analysis-and-troubleshooting/trace-logging) - Configure trace handlers, loggers, and output destinations
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Expose custom application statistics


# Logging Trace

Emit diagnostic trace from your Talon microservices using tracer objects.

## Overview

Talon provides a trace logging framework that allows you to create custom tracers in your application code and emit trace messages at various levels. Tracers are bound to loggers that control trace output destinations and levels.

## Creating Tracers

The below code snippet shows an example of creating a Tracer object called "application":

```java
@AppHAPolicy(value = AepEngine.HAPolicy.StateReplication)
public class Application {
    private static final Tracer tracer = Tracer.create("application", Level.INFO);

    @EventHandler
    final public void onMessage(Message message, Repository repository) {
        if (tracer.debug) tracer.log("onMessage entered", Level.DEBUG);
    }
}
```

The tracer is created with a default level of INFO meaning that the trace statement in the message handler which is emitted at DEBUG level will not be executed by default.

### Trace Levels

Tracers support the following trace levels (in increasing order of verbosity):

* **SEVERE** - Critical errors
* **WARNING** - Warning conditions
* **INFO** - Informational messages
* **CONFIG** - Configuration messages
* **DIAGNOSE** - Diagnostic messages
* **VERBOSE** - Verbose diagnostic messages
* **DEBUG** - Debug messages

### Conditional Tracing

For performance, always check the trace level before emitting trace at levels below INFO:

```java
if (tracer.debug) tracer.log("Debug message", Level.DEBUG);
if (tracer.verbose) tracer.log("Verbose message", Level.VERBOSE);
```

This ensures that expensive string operations or method calls are only executed when the trace level is enabled.

## Logging Trace Messages

Once you've created a tracer, use its `log()` method to emit trace:

```java
// Simple trace message
tracer.log("Processing order: " + orderId, Level.INFO);

// Conditional debug trace
if (tracer.debug) {
    tracer.log("Order details: " + order.toString(), Level.DEBUG);
}

// Error trace
tracer.log("Failed to process order: " + orderId, Level.SEVERE);
```

## Configuring Trace Output

To enable trace output and control where trace is logged, you need to configure:

1. **Trace Levels** - Control which trace messages are emitted
2. **Loggers** - Named entities that control trace output
3. **Handlers** - Destinations for trace output (stdout, files, network, etc.)

For complete information on configuring trace levels, loggers, and handlers, see [Trace Logging Configuration](/talon/operating-applications/analysis-and-troubleshooting/trace-logging).

### Quick Configuration Example

To enable debug level for the "application" tracer and bind it to both console and file output:

**DDL Configuration:**

```xml
<model xmlns="http://www.neeveresearch.com/schema/x-ddl"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <env>
    <!-- Configuration of the application tracer and logger -->
    <application>
      <trace>debug</trace>
      <logger>
        <handlers>applog,stdout</handlers>
      </logger>
    </application>

    <!-- Handler Configuration -->
    <nv>
      <logger>
        <handler>
          <stdout>stdout://</stdout>
          <applog>file://filename=/tmp/app.log&amp;count=2</applog>
        </handler>
      </logger>
    </nv>
  </env>
</model>
```

**System Properties:**

```bash
-Dapplication.trace=debug
-Dapplication.logger.handlers=applog,stdout
-Dnv.logger.handler.stdout=stdout://
-Dnv.logger.handler.applog=file://filename=/tmp/app.log&count=2
```

## Related Topics

* [Trace Logging Configuration](/talon/operating-applications/analysis-and-troubleshooting/trace-logging) - Complete guide to configuring trace handlers, loggers, and output
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Expose custom application statistics

## Next Steps

1. Create tracers for different components in your application
2. Add trace statements at appropriate levels throughout your code
3. Configure trace levels and handlers for development and production environments
4. Use conditional tracing for performance-sensitive code paths


# Configuring the Runtime

This section covers **development-time configuration** of the Talon runtime environment. These are settings you specify in your DDL (configuration XML), annotations, and system properties during development, before deployment.

## Scope of This Section

**Development-Time Configuration** includes:

* DDL parameters that control runtime behavior
* Settings specified in your `config.xml` during development
* Annotation-driven configuration in your code
* System properties that affect runtime operation

This is distinct from **runtime operations** covered in [Operating Applications](/talon/operating-applications), which focuses on administering, monitoring, and troubleshooting deployed microservices.

## Configuration Topics

### Message Flow

* [**Message Flow**](/talon/developing-applications/configuring-the-runtime/message-flow) - Configure message flow behavior
  * [Duplicate Detection](/talon/developing-applications/configuring-the-runtime/message-flow/duplicate-detection) - Configure duplicate message detection

### Runtime Behavior

* [**Configuring Adaptive Batching**](/talon/developing-applications/configuring-the-runtime/transactions/adaptive-batching) - Configure transaction batching for throughput optimization

### Threading and Performance

* [**Configuring Threading**](/talon/developing-applications/configuring-the-runtime/threading) - Configure disruptors, thread affinitization, and NUMA optimization

### Administration and Monitoring

* [**Configuring Administration**](/talon/developing-applications/configuring-the-runtime/administration) - Configure Admin over SMA for remote administration
* [**Configuring Monitoring**](/talon/developing-applications/configuring-the-runtime/monitoring) - Configure heartbeats, statistics collection, and telemetry

### Logging

* **Configuring Logging** - Configure trace logging and log levels (see [Trace Logging](/talon/operating-applications/analysis-and-troubleshooting/trace-logging))

## Development vs Operations

| Development-Time (This Section)               | Runtime Operations                                   |
| --------------------------------------------- | ---------------------------------------------------- |
| **When**: During development, in DDL/code     | **When**: After deployment, with running services    |
| **How**: DDL XML, annotations, properties     | **How**: Admin tools, viewing output, analyzing logs |
| **Example**: Configure heartbeat interval     | **Example**: View heartbeat statistics               |
| **Example**: Enable transaction latency stats | **Example**: Analyze transaction performance         |
| **Example**: Set up Admin over SMA            | **Example**: Use admin tool to manage services       |

## Related Topics

* [Operating Applications](/talon/operating-applications) - Runtime administration, monitoring, and troubleshooting
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Conceptual overview of operations architecture
* [Configuration Reference](/talon/reference/configuration) - Complete DDL elements and global properties reference

## Next Steps

1. Review the configuration topics relevant to your use case
2. Add configuration to your `config.xml` during development
3. Deploy your microservice
4. Use [Operating Applications](/talon/operating-applications) to manage and monitor at runtime


# Message Flow

## Overview

Message flow configuration controls how messages move through your Talon microservice, from receipt through processing to sending. Proper configuration of message flow features ensures reliable, ordered, and duplicate-free message processing.

This section covers runtime configuration options that affect message flow behavior. For conceptual information about message processing, see [Message Processing](/talon/developing-applications/authoring-user-code/message-processing).

## What You Can Configure

Message flow encompasses several aspects of how messages are handled:

### Duplicate Detection

Configure sequence number-based duplicate detection to ensure exactly-once processing semantics:

* Enable/disable duplicate checking per channel
* Configure sequence number windows
* Set persistence options for sequence tracking
* Handle sequence resets and rollovers

See [Duplicate Detection](/talon/developing-applications/configuring-the-runtime/message-flow/duplicate-detection) for complete configuration reference.

### Message Ordering (Future)

Configuration for message ordering guarantees:

* Ordered delivery within channels
* Key-based ordering
* Sequence number validation

*Note: Additional message flow topics will be added as the documentation expands.*

## Configuration Hierarchy

Message flow settings are configured in your DDL under the engine's messaging section:

```xml
<app name="my-service" mainClass="com.example.MyApp">
  <messaging>
    <buses>
      <bus name="orders-bus">
        <channels>
          <channel name="orders" join="true">
            <!-- Duplicate detection configuration -->
            <duplicateDetection enabled="true"/>
          </channel>
        </channels>
      </bus>
    </buses>
  </messaging>
</app>
```

## How Message Flow Works

Understanding the message processing flow helps configure these features effectively:

1. **Message Receipt**: Message arrives from messaging backbone
2. **Duplicate Check**: If enabled, sequence number checked against tracking window
3. **Handler Dispatch**: Message routed to appropriate handler based on type
4. **Business Logic**: Handler executes, updating state and sending messages
5. **Sequence Update**: If duplicate detection enabled, sequence number recorded
6. **Transaction Commit**: Changes committed atomically with consensus

See [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) for detailed flow diagrams.

## Common Configuration Patterns

### Exactly-Once Processing

Enable duplicate detection for channels where exactly-once semantics are critical:

```xml
<channel name="orders" join="true">
  <duplicateDetection enabled="true"/>
</channel>
```

### High-Throughput Channels

For channels where duplicates are acceptable or handled at application level, disable duplicate checking for maximum performance:

```xml
<channel name="market-data" join="true">
  <duplicateDetection enabled="false"/>
</channel>
```

## Related Topics

### Message Processing Concepts

* [Message Processing](/talon/concepts-and-architecture/microservice-operation/message-processing) - How messages flow through the engine
* [Transactions](/talon/concepts-and-architecture/transactions) - Transaction boundaries and guarantees

### Developer Guidance

* [Detecting Duplicates](/talon/developing-applications/authoring-user-code/message-processing/detecting-duplicates) - Understanding duplicate detection from developer perspective
* [Handling Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages) - Writing message handlers
* [Sending Messages](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/sending-messages) - Sending messages with sequence numbers

### Configuration Reference

* [Configuration](/talon/reference/configuration) - Complete DDL reference for all message flow settings

## Best Practices

1. **Enable duplicate detection for business-critical channels**: Ensure exactly-once semantics for orders, trades, and other critical messages
2. **Disable for high-volume market data**: When duplicate checking overhead is unacceptable and duplicates can be handled at application level
3. **Configure sequence persistence**: Enable sequence number persistence for channels that need recovery after cold start
4. **Monitor sequence gaps**: Use per-transaction statistics to detect and investigate sequence number gaps
5. **Test failover behavior**: Verify message retransmission and duplicate filtering work correctly during failover scenarios

## See Also

* [Transactions](/talon/developing-applications/configuring-the-runtime/transactions) - Configure transaction batching and commit behavior
* [Threading](/talon/developing-applications/configuring-the-runtime/threading) - Configure threads that process messages
* [Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Monitor message processing statistics


# Duplicate Detection

This page describes how to configure duplicate detection for inbound messages. For conceptual information about how duplicate detection works, see [Detecting Duplicates](/talon/developing-applications/authoring-user-code/message-processing/detecting-duplicates).

## Configuration

Duplicate detection is controlled by the `performDuplicateChecking` configuration parameter on the microservice:

```xml
<app name="order-processor" mainClass="com.example.OrderProcessor">
  <messaging>
    <!-- messaging config -->
  </messaging>

  <!-- Enable duplicate detection (default: true) -->
  <performDuplicateChecking>true</performDuplicateChecking>
</app>
```

### Default Behavior

By default, `performDuplicateChecking` is set to `true`, meaning duplicate detection is enabled. When enabled:

* The AEP Engine tracks sequence numbers for each bus+channel+qos combination
* Messages with sequence numbers ≤ previously received sequence numbers are discarded
* Duplicate messages are **not** dispatched to application handlers
* The `NumDupMsgsRcvd` statistic is incremented for each duplicate detected

## Disabling Duplicate Detection

You may want to disable duplicate detection if:

* Your application is **tolerant of duplicates** and has its own duplicate detection logic
* Your application performs **idempotent operations** where processing the same message multiple times has no adverse effects
* You want to implement **custom duplicate detection** using application-specific business keys rather than sequence numbers

To disable duplicate detection:

```xml
<app name="order-processor" mainClass="com.example.OrderProcessor">
  <messaging>
    <!-- messaging config -->
  </messaging>

  <!-- Disable duplicate detection -->
  <performDuplicateChecking>false</performDuplicateChecking>
</app>
```

{% hint style="warning" %}
Disabling duplicate detection means your application handlers will receive duplicate messages if the underlying message bus delivers them. Ensure your application logic can handle this correctly.
{% endhint %}

## See Also

* [Detecting Duplicates](/talon/developing-applications/authoring-user-code/message-processing/detecting-duplicates) - Conceptual overview including prerequisites and monitoring


# Transactions

## Overview

Transaction configuration controls how your Talon microservice batches and commits work. Proper transaction tuning balances latency (time to commit individual transactions) against throughput (number of transactions per second).

This section covers runtime configuration for transaction batching and commit behavior. For conceptual understanding of how transactions work, see [Transactions](/talon/concepts-and-architecture/transactions).

## What You Can Configure

Transaction configuration includes:

### Adaptive Batching

Configure the engine to adaptively batch multiple message processing transactions into a single commit:

* Enable/disable adaptive batching
* Set maximum batch size (ceiling)
* Configure batch window timing
* Tune for latency vs throughput trade-offs

See [Adaptive Batching](/talon/developing-applications/configuring-the-runtime/transactions/adaptive-batching) for complete configuration reference.

### Transaction Behavior (Future)

Additional transaction configuration topics:

* Transaction timeouts
* Commit flush behavior
* Checkpoint frequency

*Note: Additional transaction topics will be added as the documentation expands.*

## Configuration Hierarchy

Transaction settings are configured in your DDL under the engine's transaction section:

```xml
<app name="my-service" mainClass="com.example.MyApp">
  <transactions>
    <adaptiveCommitBatchCeiling>100</adaptiveCommitBatchCeiling>
  </transactions>
</app>
```

## How Transactions Work

Understanding transaction boundaries helps configure batching effectively:

### Without Adaptive Batching

Each message processed in its own transaction:

1. Message arrives
2. Handler executes
3. Transaction commits immediately
4. Next message processed

**Characteristics**:

* Lowest latency (microseconds to commit)
* Lower throughput (commit overhead per message)
* Ideal for latency-sensitive applications

### With Adaptive Batching

Multiple messages batched into single commit:

1. Messages arrive rapidly
2. Multiple handlers execute
3. Batch commits when ceiling reached or window expires
4. All messages acknowledged together

**Characteristics**:

* Higher throughput (amortize commit overhead)
* Slightly higher latency (wait for batch)
* Ideal for high-volume applications

See [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) for detailed transaction flow diagrams.

## Common Configuration Patterns

### Low-Latency Trading System

Disable batching for minimum commit latency:

```xml
<app name="trading-engine">
  <transactions>
    <!-- No batching - each message commits immediately -->
    <adaptiveCommitBatchCeiling>1</adaptiveCommitBatchCeiling>
  </transactions>
</app>
```

**Use when**:

* Sub-millisecond latency is critical
* Message arrival rate is moderate
* Each message must be processed ASAP

### High-Throughput Order Processor

Enable batching for maximum throughput:

```xml
<app name="order-processor">
  <transactions>
    <!-- Batch up to 100 messages per commit -->
    <adaptiveCommitBatchCeiling>100</adaptiveCommitBatchCeiling>
  </transactions>
</app>
```

**Use when**:

* Throughput is more important than latency
* Messages arrive in bursts
* Slight latency increase (milliseconds) is acceptable

### Balanced Configuration

Moderate batching for balanced performance:

```xml
<app name="order-router">
  <transactions>
    <!-- Batch up to 10 messages per commit -->
    <adaptiveCommitBatchCeiling>10</adaptiveCommitBatchCeiling>
  </transactions>
</app>
```

**Use when**:

* Need both good latency and throughput
* Message arrival rate varies
* Want automatic adaptation to load

## Performance Considerations

### Latency Impact

Adaptive batching increases average latency by waiting for batch to fill:

* **Batch size 1**: \~100 microseconds per message
* **Batch size 10**: \~200-500 microseconds per message
* **Batch size 100**: \~1-2 milliseconds per message

The first message in a batch sees the most latency increase.

### Throughput Gains

Batching amortizes commit overhead across multiple messages:

* **No batching**: \~10,000 messages/second
* **Batch size 10**: \~50,000 messages/second
* **Batch size 100**: \~200,000 messages/second

*Note: Actual numbers depend on message complexity, state size, and hardware.*

### Adaptive Behavior

The engine automatically adjusts batch size based on message arrival rate:

* **Low arrival rate**: Commits immediately (effective batch size 1)
* **High arrival rate**: Batches up to ceiling
* **Bursty traffic**: Adapts dynamically

This provides good latency during quiet periods and good throughput during bursts.

## Monitoring Transaction Performance

Use engine statistics to monitor transaction behavior:

* `TxnCount` - Number of transactions committed
* `TxnBatchCount` - Number of messages in each transaction
* `TxnLatency` - Time from message receipt to commit

See [Engine Statistics](/talon/developing-applications/configuring-the-runtime/monitoring/engine-statistics) for complete metrics reference.

## Related Topics

### Transaction Concepts

* [Transactions](/talon/concepts-and-architecture/transactions) - How transactions work in Talon
* [Cluster Consensus](/talon/concepts-and-architecture/microservice-operation/cluster-consensus) - Transaction commit with consensus

### Developer Guidance

* [Controlling Transactions](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions) - Programmatic transaction control
* [Using Savepoints](/talon/developing-applications/authoring-user-code/message-processing/processing-messages/handling-messages/controlling-transactions/using-savepoints) - Transaction savepoints in handlers

### Configuration Reference

* [Configuration](/talon/reference/configuration) - Complete DDL reference for transaction settings

## Best Practices

1. **Start with default (no batching)**: Begin with `adaptiveCommitBatchCeiling="1"` and measure baseline performance
2. **Increase ceiling for throughput**: If throughput is bottleneck, gradually increase ceiling (10, 25, 50, 100)
3. **Monitor latency distribution**: Track P50, P99, and P999 latencies to understand batching impact
4. **Test under realistic load**: Use production-like message rates and patterns to tune batching
5. **Consider consensus model**: Event Sourcing benefits more from batching than State Replication
6. **Account for replication overhead**: Higher batch sizes increase state delta sizes in State Replication

## See Also

* [Message Flow](/talon/developing-applications/configuring-the-runtime/message-flow) - Configure message processing behavior
* [Threading](/talon/developing-applications/configuring-the-runtime/threading) - Configure threads that process transactions
* [Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Monitor transaction statistics and performance


# Adaptive Batching

[Adaptive batching](/talon/concepts-and-architecture/transactions#adaptive-batching) is configured by setting the engine's `adaptiveCommitBatchCeiling` configuration parameter to a non-zero value. The following illustrates how this is done.

```xml
<apps>
  <app name="MyApp" className="com.foo.MyApp">
    <adaptiveCommitBatchCeiling>32</adaptiveCommitBatchCeiling>
  </app>
</app>
```

The `adaptiveCommitBatchCeiling` is set to 0 by default which disables adaptive batching. Configuring the Talon runtime to optimize for throughput i.e. set `nv.optimizefor=throughput` automatically sets the adaptive batch ceiling is set to 64 unless otherwise explicitly configured otherwise.


# Administration

## Overview

Administration configuration enables remote monitoring and management of your Talon XVMs via messaging infrastructure. By configuring administrative capabilities during development, you enable operators to manage microservices at runtime without requiring direct access to the servers.

This section covers development-time configuration for administration features. For information about using admin tools at runtime, see [Administration](/talon/operating-applications/administration).

## What You Can Configure

Administrative configuration includes:

### Admin over SMA

Configure remote administration via the Simple Messaging API:

* Enable/disable admin message processing
* Configure admin message channels
* Set administrative command permissions
* Define admin message routing

See [Admin over SMA](/talon/developing-applications/configuring-the-runtime/administration/admin-over-sma) for complete configuration reference.

### Administrative Features (Future)

Additional administration topics will be added:

* Custom administrative commands
* Administrative event subscriptions
* Admin security and authentication

*Note: Additional administration topics will be added as the documentation expands.*

## Configuration Hierarchy

Administration settings are configured in your DDL:

```xml
<app name="my-service" mainClass="com.example.MyApp">
  <admin enabled="true">
    <adminOverSMA enabled="true">
      <adminChannelName>admin-commands</adminChannelName>
      <adminResponseChannelName>admin-responses</adminResponseChannelName>
    </adminOverSMA>
  </admin>
</app>
```

## How Administration Works

Talon's administration framework allows remote management without SSH or direct server access:

### Admin Command Flow

1. **Operator sends command**: Admin tool sends command message to admin channel
2. **XVM receives**: XVM listening on admin channel receives command
3. **Command executes**: XVM processes administrative command (stats, config, shutdown, etc.)
4. **Response returned**: Result sent back on admin response channel
5. **Operator sees result**: Admin tool displays command output

### Security Considerations

Admin channels should be secured:

* Use private messaging infrastructure
* Implement channel access controls
* Consider message encryption for sensitive commands
* Audit administrative actions via trace logging

## Common Configuration Patterns

### Development Environment

Enable full admin access for local development:

```xml
<app name="dev-service">
  <admin enabled="true">
    <adminOverSMA enabled="true">
      <adminChannelName>dev-admin</adminChannelName>
      <adminResponseChannelName>dev-admin-response</adminResponseChannelName>
    </adminOverSMA>
  </admin>
</app>
```

### Production Environment

Configure secure admin channels:

```xml
<app name="prod-service">
  <admin enabled="true">
    <adminOverSMA enabled="true">
      <adminChannelName>prod-admin-secure</adminChannelName>
      <adminResponseChannelName>prod-admin-response-secure</adminResponseChannelName>
    </adminOverSMA>
  </admin>
</app>
```

Use separate admin message bus with restricted access.

### Disabled Administration

For maximum security in locked-down environments:

```xml
<app name="locked-service">
  <admin enabled="false"/>
</app>
```

## Available Administrative Commands

Once configured, operators can use these administrative commands:

### Statistics Commands

* `stats` - Display current statistics
* `stats reset` - Reset statistics counters
* `stats dump` - Write statistics to file

### Configuration Commands

* `config show` - Display current configuration
* `config get <property>` - Get specific property value
* `config set <property> <value>` - Update runtime property

### Lifecycle Commands

* `status` - Show microservice status
* `shutdown` - Graceful shutdown
* `shutdown force` - Immediate shutdown

### Discovery Commands

* `discovery show` - Show discovered entities
* `discovery refresh` - Refresh discovery cache

See [Admin Tool](/talon/operating-applications/administration/admin-tool) for complete command reference.

## Monitoring Administrative Activity

Track administrative operations via:

### Trace Logging

Enable admin trace to log all administrative commands:

```bash
-Dnv.admin.tracelevel=INFO
```

### Statistics

Monitor admin activity with statistics:

* `AdminCommandsReceived` - Number of admin commands processed
* `AdminCommandsRejected` - Number of rejected commands
* `AdminResponsesSent` - Number of responses sent

### Audit Logging

Implement custom audit logging for sensitive operations:

```java
@AdminCommandHandler("shutdown")
public void onShutdown(ShutdownCommand cmd) {
    auditLog.warn("Shutdown command received from: " + cmd.getSource());
    // ... perform shutdown
}
```

## Best Practices

1. **Enable admin for all environments**: Admin over SMA is essential for operational visibility
2. **Use separate admin channels per environment**: Don't mix dev/test/prod admin traffic
3. **Secure admin channels**: Implement access controls on admin message topics
4. **Monitor admin activity**: Log all administrative commands for audit trail
5. **Test admin commands**: Verify admin commands work before production deployment
6. **Document admin procedures**: Create runbooks for common administrative tasks

## Related Topics

### Administration Concepts

* [Operating Model](/talon/concepts-and-architecture/operating-model) - Administration architecture and patterns

### Runtime Administration

* [Administration](/talon/operating-applications/administration) - Using admin tools at runtime
* [Admin Tool](/talon/operating-applications/administration/admin-tool) - Command-line interface
* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Remote administration concepts

### Developer Guidance

* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Custom admin commands

### Configuration Reference

* [Configuration](/talon/reference/configuration) - Complete DDL reference for admin settings

## See Also

* [Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Configure statistics and heartbeats
* [Threading](/talon/developing-applications/configuring-the-runtime/threading) - Configure threads including admin thread
* [Discovery](/talon/developing-applications/configuring-the-runtime/discovery) - Configure discovery for admin channel resolution


# Admin over SMA

This page covers configuration for Admin over SMA (Simple Messaging API), which allows monitoring and management of XVMs over messaging.

## Admin Over SMA Configuration

{% hint style="info" %}
**Since 3.10**
{% endhint %}

Admin over SMA allows administrative applications to monitor and manage XVMs over messaging. When enabled, XVMs emit heartbeat, trace, and lifecycle events over defined messaging channels for consumption by listening clients.

### Basic Configuration

At a high level, enabling Admin over SMA requires setting the following properties:

```properties
nv.discovery.descriptor=<discovery-provider>
nv.server.admin.transports=sma
nv.server.admin.bus.descriptor=<bus-connection-descriptor>
```

As long as both the admin client and the XVM use the same discovery descriptor and message bus, they will be able to communicate with one another.

{% hint style="warning" %}
**Warning**: When enabling Admin over SMA it is important to note that by default admin clients using SMA will attempt to connect to discovered XVMs. If older version XVMs or XVMs not configured for Admin over SMA are advertising themselves over the admin client's discovery address, the XVM will never respond and the admin client will see connection timeouts. To avoid this you may:

* Use [passive monitoring](#passive-monitoring-configuration) on the client side to avoid attempts to ping the XVM
* Use separate XVM discovery for XVMs that support Admin over SMA vs. those that don't
  {% endhint %}

### Discovery Configuration

Admin clients should not attempt to issue commands until an XVM has been discovered via the discovery provider. When an admin client detects that an XVM is no longer discoverable, it should stop issuing commands over SMA.

See [Discovery Configuration](/talon/developing-applications/configuring-the-runtime/discovery) for details on configuring discovery.

### Configuring Admin Clients

Administrative tools such as the [Admin Tool](/talon/operating-applications/administration/admin-tool) must be configured to enable SMA as a transport along with the admin bus connection information. Since admin clients won't always be configured via DDL, configuration is done via system/environment properties.

**Properties-Based Configuration with Bus Descriptor:**

```properties
nv.server.admin.transports=sma
nv.server.admin.sma.bus.descriptor=solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true
```

**Properties-Based Configuration with Decomposed Properties:**

Alternatively, it is possible to configure the bus descriptor in decomposed form, which can be useful when configuration properties are overridden across environments:

```properties
nv.server.admin.transports=sma
nv.server.admin.sma.bus.provider=solace
nv.server.admin.sma.bus.address=solhost
nv.server.admin.sma.bus.port=55555
nv.server.admin.sma.bus.properties.SESSION_VPN_NAME=default
nv.server.admin.sma.bus.properties.use_default_queue_name=false
nv.server.admin.sma.bus.properties.use_default_queue_name_as_default_client_id=true
nv.server.admin.sma.bus.properties.topic_starts_with_channel=false
nv.server.admin.sma.bus.properties.usejni=true
nv.server.admin.sma.bus.properties.single_session=true
```

### Configuring XVMs

XVMs can be configured using the same environment properties as clients by setting the properties in the DDL environment section:

```xml
<env>
  <nv>
    <server>
      <admin>
        <transports>sma</transports>
        <sma>
          <bus>
            <descriptor>solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true</descriptor>
          </bus>
        </sma>
      </admin>
    </server>
  </nv>
</env>
```

### Advanced XVM Bus Configuration

Using environment-based configuration is the simplest way of configuring Admin over SMA for an XVM. In cases where bus configuration is not being injected by deployment tools, it is possible to use the `<xvm>` `<admin>` element in DDL to enable admin over SMA and reference a DDL-defined bus definition.

**XVM Admin over SMA Configuration:**

To configure an XVM to use a bus named 'xvm-admin', the XVM's `<admin>` `<sma>` element can be used:

```xml
<xvms>
  <xvm name="order-processing-1" template="xvm-template">
    <admin>
      <transports>
        <sma enabled="true">
          <busName>xvm-admin</busName>
        </sma>
      </transports>
    </admin>
    <heartbeats enabled="true" interval="5s"/>
  </xvm>
</xvms>
```

**Admin Bus Configuration:**

The following bus definition can then be configured for use by the XVM:

```xml
<buses>
  <bus name="xvm-admin">
    <provider>solace</provider>
    <address>solhost</address>
    <port>55555</port>
  </bus>
</buses>
```

Admin channels (xvm-request, xvm-response, xvm-heartbeat, xvm-event, and xvm-trace) should not be configured for the bus; they are automatically created by the XVM when it is started.

With the above configuration, the `order-processing-1` XVM will create a connection to solace://solhost:55555 with a username of `order-processing-1`. It will use the following topics:

**Subscribe:**

* **xvm-request channel:** xvm-admin/order-processing-1/request

**Publish:**

* **xvm-response channel:** xvm-admin/${adminClientId}/response (where the adminClientId is substituted with that of the sending client when a response is sent)
* **xvm-heartbeats channel:** xvm-admin/order-processing-1/heartbeat (when heartbeats are enabled)
* **xvm-trace channel:** xvm-admin/order-processing-1/trace (when trace emission is enabled)
* **xvm-event channel:** xvm-admin/order-processing-1/event

{% hint style="info" %}
**Tip**: Enabling XVM heartbeats isn't strictly necessary, but in most monitoring scenarios it is desirable.
{% endhint %}

### Channel Prefix Configuration

By default, all admin channels start with the *xvm-admin* channel prefix. This prefix can be changed by setting the environment property `nv.admin.sma.channelKeyPrefix`, which can be useful for cases where it is desirable to more granularly partition admin traffic.

### Passive Monitoring Configuration

It is possible to use Admin over SMA in a purely passive monitoring capacity by setting the property:

```properties
nv.server.admin.passivemonitoringonly=true
```

With the above configuration setting, clients will throw an exception if an attempt is made to send commands to an XVM, and XVMs will not issue subscriptions on the xvm-request channel.

## Related Topics

* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Using Admin over SMA at runtime
* [Admin Tool](/talon/operating-applications/administration/admin-tool) - Command-line administrative interface
* [Configuring Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Configure heartbeat and statistics emission

## Next Steps

1. Configure discovery provider for XVM and admin clients
2. Set up message bus for admin channels
3. Enable Admin over SMA in XVM configuration
4. Configure admin tools to use SMA transport
5. Test admin operations via messaging


# Monitoring

Configure statistics collection and heartbeat emission for monitoring your Talon microservices.

## Overview

Talon automatically collects extensive statistics about message processing, transactions, and system resources. Configuration controls what statistics are collected, how frequently they're emitted, and the performance trade-offs.

Statistics must be balanced against performance impact. Many are enabled by default with minimal overhead, while detailed latency and per-transaction statistics can impact performance and require explicit enablement.

## Topics

* [**XVM Heartbeats**](/talon/developing-applications/configuring-the-runtime/monitoring/xvm-heartbeats) - Configure periodic XVM-level statistics emission
* [**Engine Statistics**](/talon/developing-applications/configuring-the-runtime/monitoring/engine-statistics) - Configure AEP engine metrics collection
* [**Memory Statistics**](/talon/developing-applications/configuring-the-runtime/monitoring/memory-statistics) - Configure memory stats collection (heap, off-heap, IO buffers, entity lifecycle)
* [**Per-Transaction Statistics**](/talon/developing-applications/configuring-the-runtime/monitoring/per-transaction-statistics) - Configure detailed transaction-level metrics

## Related Topics

* [Operating Model](/talon/concepts-and-architecture/operating-model) - Monitoring architecture
* [Monitoring](/talon/operating-applications/monitoring) - Viewing and interpreting statistics at runtime
* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Define custom application metrics


# XVM Heartbeats

Configure periodic XVM-level statistics emission from your XVM.

## Overview

XVM heartbeats provide periodic snapshots of XVM-level statistics including system metrics, thread statistics, pool usage, engine stats, and application-defined stats. Heartbeats can be emitted via messaging, logged to binary transaction logs, or traced for debugging.

Statistics collection must be balanced against performance impact. While many statistics have minimal overhead, enabling all collection options can impact latency-sensitive applications.

## Heartbeat Configuration

Heartbeats for an XVM can be enabled via DDL using the `<heartbeats>` element:

```xml
<xvms>
  <xvm name="my-xvm">
    <heartbeats enabled="true" interval="5">
      <collectNonZGStats>true</collectNonZGStats>
      <collectIndividualThreadStats>true</collectIndividualThreadStats>
      <collectSeriesStats>true</collectSeriesStats>
      <collectSeriesDatapoints>false</collectSeriesDatapoints>
      <maxTrackableSeriesValue>100000000</maxTrackableSeriesValue>
      <includeMessageTypeStats>false</includeMessageTypeStats>
      <collectPoolStats>true</collectPoolStats>
      <poolDepletionThreshold>1.0</poolDepletionThreshold>
      <logging enabled="true"></logging>
      <tracing enabled="true"></tracing>
    </heartbeats>
  </xvm>
</xvms>
```

## Configuration Settings

| Configuration Setting          | Default     | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| ------------------------------ | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `enabled`                      | false       | <p>Enable or disable XVM stats collection and heartbeat emission.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br><strong>Performance Impact</strong>: Collection of stats and emission of heartbeats can impact application performance from both a latency and throughput standpoint. For applications that are particularly sensitive to performance, it is a good idea to compare performance with and without heartbeats enabled to understand the overhead that is incurred by enabling heartbeats.<br></p></div>                                                                                                                                                                                                                                                                                                                                                                                               |
| `interval`                     | 1000        | The interval in seconds at which XVM stats will be collected and emitted                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `collectNonZGStats`            | true        | Some statistics collected by the stats collection thread require creating a small amount of garbage. This can be set to false to suppress collection of these stats                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `collectIndividualThreadStats` | true        | Indicates whether heartbeats will contain stats for each active thread in the JVM. Individual thread stats are useful for identifying busy threads and thread affinitization candidates                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| `collectSeriesStats`           | true        | Indicates whether or not series stats should be included in heartbeats                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `collectSeriesDatapoints`      | false       | <p>Indicates whether or not series stats should report the data points captured for a series statistic.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br><strong>Warning</strong>: Enabling this value includes each datapoint collected in a series in heartbeats which can make emitted heartbeats very large and slow down their collection. It is not recommended that this be run in production.<br></p></div>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `maxTrackableSeriesValue`      | 10 minutes  | The maximum value (in **microseconds**) that can be tracked for reported series histogram timings. Datapoints above this value will be downsampled to this value, but will be reflected in the max value reported in an interval                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `includeMessageTypeStats`      | false       | <p>Sets whether or not message type stats are included in heartbeats (when enabled for the app).<br><br>When <code>captureMessageTypeStats</code> is enabled for an app, the AEP engine will record select statistics on a per message type basis. Because inclusion of per message type stats can significantly increase the size of heartbeats, inclusion in heartbeats is disabled by default.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: For message type stats to be included in heartbeats, both <code>captureMessageTypeStats</code> for the app must be set to true (capture is disabled by default because recording them is costly), and <code>includeMessageTypeStats</code> must be set to true (inclusion is disabled by default because emitting them is costly).<br></p></div>                                                                                                    |
| `collectPoolStats`             | true        | Indicates whether or not pool stats are collected by the XVM                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `poolDepletionThreshold`       | 1.0         | <p>Configuration property used to set the percentage decrement at which a preallocated pool must drop to be included in an XVM heartbeat. Setting this to a value greater than 100 or less than or equal to 0 disables depletion threshold reporting.<br><br>This gives monitoring applications advanced warning if it looks like a preallocated pool may soon be exhausted. By default the depletion threshold is set to trigger inclusion in heartbeats at every 1% depletion of the preallocated count. This can be changed by specifying the configuration property <code>nv.server.stats.pool.depletionThreshold</code> to a float value between 0 and 100.<br><br><strong>Example</strong>: If a pool is preallocated with 1000 items and this property is set to 10, pool stats will be emitted for the pool each time a heartbeat occurs and the pool has dropped below a 10% threshold of the preallocated size (e.g., at 900, 800, 700, until its size reaches 0).</p> |
| `logging`                      | *See below* | <p>Configures binary logging of heartbeats.<br><br>Binary heartbeat logging provides a means by which heartbeat data can be captured in a zero garbage fashion. Collection of such heartbeats can be useful in diagnosing performance issues in running apps</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `tracing`                      | *See below* | <p>Configures trace logging of heartbeats.<br><br>Enabling textual tracing of heartbeats is a useful way to quickly capture data from XVM heartbeats for applications that aren't monitoring XVM heartbeats remotely. Textual trace of heartbeats is not zero garbage and is therefore not suitable for applications that are latency sensitive</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

## Trace Output Configuration

By default all XVM statistics tracers are disabled as trace logging is not zero garbage and introduces CPU overhead. While tracing heartbeats isn't recommended in production, enabling XVM statistics trace output can be useful for debugging and performance tuning.

```xml
<xvms>
  <xvm name="my-xvm">
    <heartbeats enabled="true" interval="5">
      <tracing enabled="true">
        <traceSysStats>true</traceSysStats>
        <traceThreadStats>true</traceThreadStats>
        <tracePoolStats>true</tracePoolStats>
        <traceAppStats>true</traceAppStats>
        <traceUserStats>true</traceUserStats>
      </tracing>
    </heartbeats>
  </xvm>
</xvms>
```

## Binary Logging Configuration

Applications that are latency sensitive might prefer to leave all tracers disabled to avoid unnecessary allocations and the associated GC activity. As an alternative, it's possible to enable logging of zero-garbage heartbeat messages to a binary transaction log:

```xml
<xvms>
  <xvm name="my-xvm">
    <heartbeats enabled="true" interval="5">
      <logging enabled="true">
         <storeRoot>/path/to/heartbeat/log/directory</storeRoot>
      </logging>
    </heartbeats>
  </xvm>
</xvms>
```

When a `storeRoot` is not set, an XVM will log heartbeats to `{XRuntime.getDataDirectory}/server-heartbeats/<xvm-name>-heartbeats.log`, which can then be queried and traced from a separate process using the [Stats Dump Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/stats-dump-tool).

{% hint style="warning" %}
**Warning**: At this time binary heartbeat logs do not support rolling collection. Consequently this mechanism is not suitable for long running application instances.
{% endhint %}

## Memory Stats Configuration

In addition to the JVM heap and non-heap memory shown in the system stats section of heartbeats, Talon can collect detailed off-heap memory, IO buffer, and ADM entity lifecycle statistics. These are included in heartbeat messages and can also be traced independently.

| Property                      | Default | Description                                                                                                                                                                                                                                     |
| ----------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `nv.memory.stats.enable`      | `false` | Enable detailed memory stats collection (off-heap, IO buffer, entity tracking). Must be set to `true` for memory stats to appear in heartbeats and standalone trace                                                                             |
| `nv.memory.stats.type.enable` | `false` | Enable per-entity-type breakdown. When enabled, memory stats include counters for each individual message, entity, embedded entity, and collection type                                                                                         |
| `nv.memory.stats.interval`    | `0`     | Standalone memory stats trace interval in seconds. When set to a value greater than 0, memory stats are traced to the `nv.memory.stats` logger independently of heartbeats. Useful for debugging memory issues without enabling full heartbeats |

These properties can be set as system properties or in DDL:

```xml
<env>
  <nv.memory.stats.enable>true</nv.memory.stats.enable>
  <nv.memory.stats.type.enable>true</nv.memory.stats.type.enable>
</env>
```

{% hint style="info" %}
**See Also**: [Memory Stats](/talon/operating-applications/monitoring/memory-statistics) for a complete reference of all memory statistics collected and their meaning.
{% endhint %}

## Related Topics

* [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats) - Viewing and interpreting heartbeat statistics at runtime
* [Memory Stats](/talon/operating-applications/monitoring/memory-statistics) - Detailed memory statistics reference
* [Engine Statistics](/talon/developing-applications/configuring-the-runtime/monitoring/engine-statistics) - Configure engine-level statistics collection
* [Stats Dump Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/stats-dump-tool) - Analyzing binary heartbeat logs
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Monitoring architecture overview

## Next Steps

1. Enable heartbeats with appropriate collection settings
2. Test performance impact under load
3. Configure binary logging for zero-garbage collection or tracing for debugging
4. Set up monitoring tools to consume heartbeats
5. Balance operational visibility against performance requirements


# Engine Statistics

Configure AEP engine statistics collection including message latencies, transaction metrics, and per-message-type statistics.

## Overview

AEP engine statistics provide detailed metrics about message processing, transaction execution, and bus activity. Most engine metrics are low overhead and always collected, but latency statistics and per-message-type stats can impact performance and are disabled by default.

## Global Statistics Configuration

An XVM collects stats that are enabled for the applications that it contains. The following global statistics can be configured via environment properties:

### Global Environment Properties

```xml
<env>
  <nv>
    <!-- Sample size configuration -->
    <stats.latencymanager.samplesize>65536</stats.latencymanager.samplesize>
    <stats.series.samplesize>10240</stats.series.samplesize>

    <!-- Global latency stats enablement -->
    <msg.latency.stats>true</msg.latency.stats>
    <ods.latency.stats>true</ods.latency.stats>
    <event.latency.stats>true</event.latency.stats>
    <msgtype.latency.stats>false</msgtype.latency.stats>

    <!-- Low-level I/O timestamps -->
    <link.network.stampiots>true</link.network.stampiots>
  </nv>
</env>
```

| Environment Property                 | Default                      | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| ------------------------------------ | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `nv.stats.latencymanager.samplesize` | `nv.stats.series.samplesize` | <p>The global default size used for capturing latencies. Latencies stats are collected in a ring buffer which is sampled by the stats thread at each collection interval.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: This should be sized large enough such that datapoints aren't missed, but not so large that it adversely affects processor cache performance.<br></p></div>                                                                                                                                                       |
| `nv.stats.series.samplesize`         | 10240                        | <p>Property that can be used to control the default sampling size for Series stats.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: If the number of datapoints collected in a stats interval exceeds this size, the computation for histographical data will be lossy. Increasing the value will reduce loss of datapoints but results in greater overhead in stats collection in terms of both memory usage and pressure on the process caches.<br></p></div>                                                                             |
| `nv.msg.latency.stats`               | false                        | <p>This global property instructs the platform to collect latency statistics for messages passing through various points in the process pipeline.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: When not enabled, many latency stats will not be available in heartbeats.<br></p></div>                                                                                                                                                                                                                                                   |
| `nv.msgtype.latency.stats`           | false                        | <p>Property that enables tracking of message latency stats on a type by type basis. When set to true, timings for each message type are individually tracked as separate stats.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br><strong>Warning</strong>: Due to their overhead, these statistics are not included in heartbeats emitted by an XVM.<br></p></div>                                                                                                                                                                                           |
| `nv.ods.latency.stats`               | false                        | Globally enables collection of application store latencies                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `nv.event.latency.stats`             | false                        | <p>Indicates whether or not event latency statistics are captured. Enabling Event latency stats record timestamps for enqueue and dequeue of events across event multiplexers, such as the AepEngine's input multiplexer queue. Enabling event latency stats is useful for determining if an engine's event multiplexer queue is backing up by recording the time that events remain on the input queue.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: These stats must be enabled in order to capture input queuing times.<br></p></div> |
| `nv.link.network.stampiots`          | false                        | <p>Instructs low level socket I/O stamp input/output times on written data.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p><br><strong>Tip</strong>: This should be enabled to capture store latencies based on wire time, or for certain latencies in the direct message bus binding.<br></p></div>                                                                                                                                                                                                                                                                 |

## Per-Engine Statistics Configuration

Latencies related to a particular microservice's transaction pipeline can be configured at the application level:

```xml
<apps>
  <app name="my-app" mainClass="com.foo.MyApp">
    <captureTransactionLatencyStats>true</captureTransactionLatencyStats>
    <captureEventLatencyStats>true</captureEventLatencyStats>
    <captureMessageTypeStats>false</captureMessageTypeStats>
    <messageTypeStatsLatenciesToCapture>c2o,o2p,mpproc,mproc,mfilt</messageTypeStatsLatenciesToCapture>
  </app>
</apps>
```

| Configuration Setting                | Default | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| ------------------------------------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `captureTransactionLatencyStats`     | false   | Property that enables collection of latency stats as messages flow through the AEP engine's transaction processing machinery.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| `captureEventLatencyStats`           | false   | <p>Property that globally enables collection of message latency stats as messages flow through the system. These statistics include latencies in the flow outside of transaction processing. For received messages these statistics include transmission, deserialization and dispatch costs. For sent messages these include serialization and transmission costs.<br><br>When set to true, timings for messages are captured as they flow through the system. Enablement of these stats is required to collect message bus latency stats. Enabling this property can increase latency due to the overhead of tracking timestamps.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `captureMessageTypeStats`            | false   | <p>Property that enables tracking of message statistics on a per message type basis.<br><br>When set to true, statistics for each message type are individually tracked.<br><br></p><div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p><br><strong>Warning</strong>: Due to their overhead, these statistics are not included in heartbeats emitted by an XVM unless <code>includeMessageTypeStats</code> is also enabled in heartbeat configuration.<br></p></div>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `messageTypeStatsLatenciesToCapture` | all     | <p>Property controlling which latency stats on a per message type basis. This property is specified as a comma separated list of values. Valid values include:<br><br>- <strong>all</strong> - Indicates that all available per message type latency stats should be collected.<br>- <strong>none</strong> - Indicates that no message type latency stats should be collected.<br>- <strong>c2o</strong> - Indicates create to offer latencies should be captured.<br>- <strong>o2p</strong> - Indicates offer to poll (input queueing time) should be captured.<br>- <strong>mfilt</strong> - Indicates that time spent in application message filters should be captured.<br>- <strong>mpproc</strong> - Indicates that time spent in the engine prior to message dispatch should be captured.<br>- <strong>mproc</strong> - Indicates that the time spent in application message handlers should be captured.<br><br>The values 'all' or 'none' may not be combined with other values.<br><br>This value only applies when captureMessageTypeStats is true. When not specified the value defaults to <strong>all</strong>.</p> |

## Message Type Specific Stats

To enable message type specific stats and include them in heartbeats:

```xml
<apps>
  <app name="MyApp">
    <captureMessageTypeStats>true</captureMessageTypeStats>
  </app>
</apps>

<xvms>
  <xvm name="MyXVM">
    <heartbeats enabled="true" interval="5">
      <includeMessageTypeStats>true</includeMessageTypeStats>
    </heartbeats>
  </xvm>
</xvms>
```

## Statistics Output Threads (Development/Testing Only)

The following output threads can be enabled to trace individual types of statistics, which is useful for testing and performance tuning. Enabling these output threads *is not required* for collecting stats. Statistics trace output is not zero garbage, so in a production scenario it usually makes more sense to collect stats via [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats), which emits zero garbage heartbeats. For standalone stats output format details, see [Engine Stats - Standalone Statistics Output Format](/talon/operating-applications/monitoring/engine-statistics#standalone-statistics-output-format). For engine stats in heartbeat trace, see [XVM Heartbeats - Engine Stats](/talon/operating-applications/monitoring/xvm-heartbeats#engine-stats).

{% hint style="info" %}
When an AepEngine is running inside of an XVM (the most common case), engine statistics are included in XVM heartbeats and should be traced using the XVM tracing facilities. The trace threads described below should not be enabled when running within an XVM as collection by the trace threads and XVM Stats collector thread can interfere with one another.

See [Tracing Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats#heartbeat-trace-output)
{% endhint %}

| Configuration Setting                             | Default | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| ------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| nv.aep.\<engine>.stats.interval                   | 0       | <p>The interval (in <strong>seconds</strong>) at which engine stats will be traced for a given engine.<br><br>Can be set to a positive integer to indicate the period in <strong>seconds</strong> at which the engine's stats dump thread will dump recorded engine statistics. Setting a value of 0 disables creation of the stats thread.<br><br>When enabled, engine stats are traced to the logger 'nv.aep.engine.stats' at a level of <code>Tracer.Level.INFO</code>; therefore, to see dumped stats, a trace level of 'nv.aep.engine.stats.trace=info' must be enabled.<br><br><strong>NOTE</strong>: Disabling the engine stats thread only stops stats from being periodically <strong>traced</strong>. It does not stop the engine from collecting stats; stats can still be collected by an external thread (such as the XVM which reports the stats in XVM heartbeats). In other words, enabling the stats thread is not a prerequisite for collecting stats, and disabling the stats reporting thread does not stop them from being collected.<br><br><strong>NOTE</strong>: While collection of engine stats is a zero garbage operation, tracing engine stats is not zero garbage when performed by this stats thread. For latency sensitive apps, it is recommended to run in an XVM which can collect engine stats and report them in heartbeats in a zero garbage fashion.</p> |
| nv.aep.\<engine>.sysstats.interval                | 0       | <p>The interval (in <strong>seconds</strong>) at which engine sys stats will be reported. Set to <strong>0</strong> (the default) to completely disable sys stats tracing for a given engine.<br><br>In most cases, AEP sys stats will not be used and system level stats would be recorded in the XVM Statistics from which an AEPEngine is running.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| nv.event.mux.\<name>.stats.interval               | 0       | <p>The interval (in <strong>seconds</strong>) at which multiplexer stats will be traced.<br><br>Multiplexer stats can also be reported as part of the overall engine stats from the engine stats thread, so there is no need to set this to a non-zero value if nv.aep.\<engine>.stats.interval is greater than zero.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| nv.msg.latency.stats.interval                     | 0       | <p>The interval (in seconds) at which message latency stats are traced.<br><br>This setting has no effect if nv.msg.latency.stats is false. This allows granular tracing of just message latency stats on a per bus basis. Message latency stats can also be reported as part of the overall engine stats from the engine stats thread, so there is no need to set this to a non-zero value if nv.aep.\<engine>.stats.interval is greater than zero.</p>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| nv.aep.busmanager.\<engine>.\<bus>.stats.interval | 0       | The interval (in **seconds**) at which bus stats will be traced. Bus stats are reported as part of the overall engine stats from the engine stats thread, so there is no need to set this to a non-zero value if nv.aep.\<engine>.stats.interval is greater than zero. When engine stats output is disabled this can be used to trace only bus stats for a particular message bus.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

## Related Topics

* [Engine Stats](/talon/operating-applications/monitoring/engine-statistics) - Understanding and interpreting engine metrics and standalone stats output format
* [XVM Heartbeats - Engine Stats](/talon/operating-applications/monitoring/xvm-heartbeats#engine-stats) - Engine stats in heartbeat trace output
* [XVM Heartbeats](/talon/developing-applications/configuring-the-runtime/monitoring/xvm-heartbeats) - Configure XVM-level heartbeat emission
* [Per-Transaction Statistics](/talon/developing-applications/configuring-the-runtime/monitoring/per-transaction-statistics) - Configure detailed transaction-level metrics
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Monitoring architecture overview

## Next Steps

1. Determine which latency statistics are needed for your use case
2. Configure global latency stats if detailed performance metrics are required
3. Enable per-engine statistics for transaction and event latencies
4. Test performance impact of statistics collection under load
5. Balance operational visibility against performance requirements


# Memory Statistics

Configure memory statistics collection covering JVM heap, off-heap (native) memory, IO buffers, and ADM entity lifecycle tracking.

## Overview

Talon can track detailed memory usage statistics including JVM heap and non-heap usage, off-heap (native) memory allocations, IO buffer lifecycle, and ADM entity pooling. Memory stats collection is disabled by default because it adds some overhead. When enabled, memory stats are included in XVM heartbeats and can also be output independently via standalone trace logging.

## Enabling Memory Stats

To enable memory stats collection, set the `nv.memory.stats.enable` system property to `true`:

```bash
-Dnv.memory.stats.enable=true
```

Or in DDL:

```xml
<env>
  <nv.memory.stats.enable>true</nv.memory.stats.enable>
</env>
```

To disable memory stats, remove the property or set it to `false`.

### Per-Entity-Type Breakdown

By default, entity stats are reported in aggregate across all entity types. To additionally get a per-entity-type breakdown, enable the type-level property:

```bash
-Dnv.memory.stats.enable=true -Dnv.memory.stats.type.enable=true
```

Or in DDL:

```xml
<env>
  <nv.memory.stats.enable>true</nv.memory.stats.enable>
  <nv.memory.stats.type.enable>true</nv.memory.stats.type.enable>
</env>
```

## Standalone Trace Output

Memory stats can be output periodically to a trace logger independently of XVM heartbeats. This is useful for standalone applications or debugging memory issues outside of the heartbeat cycle.

Set the `nv.memory.stats.interval` system property to the desired output interval in seconds:

```bash
-Dnv.memory.stats.enable=true -Dnv.memory.stats.interval=5
```

Or in DDL:

```xml
<env>
  <nv.memory.stats.enable>true</nv.memory.stats.enable>
  <nv.memory.stats.interval>5</nv.memory.stats.interval>
</env>
```

When enabled, a background thread (`X-Stats-Printer [com.neeve.memory.stats]`) periodically collects and traces memory stats to the `nv.memory.stats` logger at `INFO` level.

## Configuration Properties Reference

| Property                      | Default | Description                                                                        |
| ----------------------------- | ------- | ---------------------------------------------------------------------------------- |
| `nv.memory.stats.enable`      | `false` | Enable memory stats collection. Must be `true` for any memory stats to be recorded |
| `nv.memory.stats.type.enable` | `false` | Enable per-entity-type breakdown of entity stats                                   |
| `nv.memory.stats.interval`    | `0`     | Standalone trace output interval in seconds. `0` disables standalone trace         |

## Related Topics

* [Memory Stats](/talon/operating-applications/monitoring/memory-statistics) - Understanding and interpreting memory statistics at runtime
* [XVM Heartbeats](/talon/developing-applications/configuring-the-runtime/monitoring/xvm-heartbeats) - Configure XVM-level heartbeat emission
* [Engine Statistics](/talon/developing-applications/configuring-the-runtime/monitoring/engine-statistics) - Configure AEP engine metrics collection
* [Operating Model](/talon/concepts-and-architecture/operating-model) - Monitoring architecture overview


# Per-Transaction Statistics

Configure detailed transaction-level statistics collection for in-depth performance analysis.

{% hint style="warning" %}
**Performance Impact**: Users are advised to carefully test the performance impact of enabling per transaction stats particularly when operating under load, as collection and reporting of these stats can be quite costly in terms of CPU usage, disk write bandwidth, and disk space usage.
{% endhint %}

## Overview

Per-transaction stats provide detailed statistics for each transaction and message at the cost of greater overhead. Unlike aggregated engine statistics, per-transaction stats record timestamps and metrics for every transaction, allowing granular analysis of performance outliers and bottlenecks.

## Configuration

Per-transaction stats collection requires enabling both global latency collection and per-transaction capture:

```xml
<env>
  <nv>
    <!-- globally enable message latency stats -->
    <msg.latency.stats>true</msg.latency.stats>
    <!-- globally enable ODS store stats collection -->
    <ods.latency.stats>true</ods.latency.stats>
    <!-- Enable low level I/O timestamps-->
    <link.network.stampiots>true</link.network.stampiots>
  </nv>
</env>

<apps>
  <app name="processor" mainClass="com.neeve.talon.starter.Application">
    <!-- Enable transaction latency stats collection -->
    <captureTransactionLatencyStats>true</captureTransactionLatencyStats>
    <!-- Capture transaction stats on a per transaction basis -->
    <capturePerTransactionStats>true</capturePerTransactionStats>
    <!-- Configure Per Transaction Stats Logger -->
    <perTransactionStatsLogging policy="UseDedicated">
      <flushOnCommit>true</flushOnCommit>
      <detachedWrite enabled="true">
        <queueOfferStrategy>SingleThreaded</queueOfferStrategy>
        <queueWaitStrategy>Blocking</queueWaitStrategy>
        <queueDrainerCpuAffinityMask>0</queueDrainerCpuAffinityMask>
      </detachedWrite>
    </perTransactionStatsLogging>
  </app>
</apps>
```

With the above configuration, the application will create a `processor.txnstats.log` in the application's data directory. At the end of each transaction, an `AepMonTransactionStatsMessage` will be saved to this transaction log file which contains the captured stats for that transaction.

## Configuration Parameters

| Configuration Setting         | Description                                                                                                       |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `capturePerTransactionStats`  | Enables per-transaction statistics collection. Requires `captureTransactionLatencyStats` to also be enabled.      |
| `perTransactionStatsLogging`  | Configures the dedicated binary transaction log for per-transaction stats. Policy should be set to `UseDedicated` |
| `flushOnCommit`               | Whether to flush stats to disk after each transaction commit. Set to `true` for immediate persistence.            |
| `detachedWrite`               | Configures detached writing to offload I/O from the dispatcher thread. Should generally be enabled.               |
| `queueOfferStrategy`          | Strategy for offering messages to the detached write queue. Use `SingleThreaded` for single-threaded dispatchers. |
| `queueWaitStrategy`           | Wait strategy for the detached write disruptor. `Blocking` is appropriate for non-latency-critical logging.       |
| `queueDrainerCpuAffinityMask` | CPU affinity mask for the detached write thread.                                                                  |

## Prerequisites

Per-transaction stats collection requires the following global settings to be enabled:

* `nv.msg.latency.stats=true` - Enables message latency timestamps
* `nv.ods.latency.stats=true` - Enables store latency timestamps
* `nv.link.network.stampiots=true` - Enables low-level I/O timestamps
* `captureTransactionLatencyStats=true` - Enables transaction latency collection

## Performance Considerations

Per-transaction stats collection has significant performance impact:

* **CPU Usage**: Capturing and recording timestamps for every transaction
* **Disk I/O**: Writing stats for every transaction to disk
* **Disk Space**: Stats logs can grow rapidly under high transaction rates
* **Memory**: Buffering stats before writing

Always test the performance impact in a non-production environment before enabling in production.

## Related Topics

* [Per Transaction Stats](/talon/operating-applications/monitoring/per-transaction-statistics) - Working with per-transaction statistics logs at runtime
* [Engine Statistics](/talon/developing-applications/configuring-the-runtime/monitoring/engine-statistics) - Configure engine-level statistics collection
* [XVM Heartbeats](/talon/developing-applications/configuring-the-runtime/monitoring/xvm-heartbeats) - Configure XVM-level heartbeat emission
* [Transactions](/talon/concepts-and-architecture/transactions) - Transaction concepts

## Next Steps

1. Understand the performance impact of per-transaction stats
2. Enable global latency stats collection
3. Configure per-transaction stats logging in DDL
4. Test performance under load in non-production environment
5. Use [TransactionStatsLogTool](/talon/operating-applications/monitoring/per-transaction-statistics#the-transactionstatslogtool) to analyze collected stats
6. Disable after collecting necessary data


# Threading

Configure threading, disruptors, and CPU affinity for optimal performance in your Talon microservices.

## Overview

Talon's threading model is built on the single writer principle with detached threads for infrastructure work. Configuration allows you to tune disruptor parameters, pin threads to specific CPU cores, and optimize for NUMA architectures.

Proper threading configuration is critical for achieving ultra-low latency and maximum throughput.

## Topics

* [**Thread Reference**](/talon/developing-applications/configuring-the-runtime/threading/thread-reference) - Complete reference of all Talon threads and their configuration options
* [**Disruptors**](/talon/developing-applications/configuring-the-runtime/threading/disruptors) - Configure LMAX disruptor ring buffers for inter-thread communication
* [**Thread Affinitization**](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) - Pin threads to CPU cores for cache locality and NUMA optimization

## Related Topics

* [Threading Model](/talon/concepts-and-architecture/threading-model) - Threading architecture and design rationale
* [Operating Model](/talon/concepts-and-architecture/operating-model) - How threading fits into operations


# Disruptors

Configure LMAX Disruptor ring buffers for inter-thread communication in Talon microservices.

{% hint style="info" %}
**Prerequisites**: Before diving into configuration, review the [Threading Model](/talon/concepts-and-architecture/threading-model) page to understand the architectural concepts and design rationale behind Talon's threading architecture.
{% endhint %}

## Overview

Talon uses [LMAX Disruptors](https://github.com/LMAX-Exchange/disruptor) to pass data between critical threads in the processing pipeline. Disruptors provide ultra-low latency inter-thread communication using ring buffers and wait strategies optimized for different performance characteristics.

Throughout Talon configuration you'll see disruptor configuration that looks like:

```xml
<persistence enabled="true">
  <detachedPersist enabled="true">
    <queueDepth></queueDepth>
    <queueOfferStrategy></queueOfferStrategy>
    <queueWaitStrategy></queueWaitStrategy>
    <queueDrainerCpuAffinityMask></queueDrainerCpuAffinityMask>
  </detachedPersist>
</persistence>
```

## Disruptor Parameters

| Parameter                     | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `queueDepth`                  | The size of the ring buffer. This knob controls the size of the ring buffer. It is best to choose a power of 2 for ring buffer. The buffer should be sized large enough to absorb spikes in microservice traffic without blocking the offering thread, but otherwise should generally be kept small enough to keep the amount of active data in the pipeline small enough to avoid taxing CPU caches. The default size for most disruptors is 1024.                                                                                                                                                                                                                                                      |
| `queueWaitStrategy`           | Controls how the thread draining events from the ring buffer waits for more events. One of `BusySpin`, `Yielding`, `Sleeping`, `Blocking`. For microservices that want the lowest latency possible using BusySpin causes the draining thread to spin without signaling to the OS that it should be context switched which avoids jitter. This policy is most appropriate when the number of cores available in the machine is adequate for each reader to occupy its own core. Otherwise, a Yielding wait strategy can be used. Both BusySpin and Yielding are CPU intensive and are most appropriate for microservices where performance is critical and run on hardware dedicated to the microservice. |
| `queueDrainerCpuAffinityMask` | Controls the CPU to which to affinitize the draining thread. For BusySpin or Yielding policies, affinitizing threads can further reduce jitter. See [Thread Affinitization](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) for details.                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `queueOfferStrategy`          | This can be used to override the offer strategy used to manage concurrency when offering elements to the ring buffer. **Warning**: In general, microservices should not change this property as the platform will choose a sensible default.                                                                                                                                                                                                                                                                                                                                                                                                                                                             |

## Auto-Tuning of Disruptor Wait Strategies

When the `nv.optimizefor` environment property is set to `latency` or `throughput`, disruptors in the critical path are automatically set to BusySpin and Yielding respectively unless explicitly configured otherwise via configuration.

You can set the environment property:

```xml
<env>
  <nv.optimizefor>latency</nv.optimizefor>
  <nv.conservecpu>true</nv.conservecpu>
</env>
```

## Related Topics

* [Threading Model](/talon/concepts-and-architecture/threading-model) - Architectural concepts and design rationale
* [Thread Reference](/talon/developing-applications/configuring-the-runtime/threading/thread-reference) - Complete reference of all Talon threads
* [Thread Affinitization](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) - Pin threads to CPU cores for optimal performance
* [DDL Reference](/talon/reference/configuration) - Complete DDL syntax reference

## Next Steps

1. Review the [Threading Model](/talon/concepts-and-architecture/threading-model) to understand disruptor usage
2. Identify which disruptors are in your microservice's critical path
3. Configure wait strategies based on your performance requirements
4. Consider [Thread Affinitization](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) for disruptor drainer threads


# Thread Reference

Complete reference of all threads created by Talon microservices, including configuration examples for affinitizing critical threads.

{% hint style="info" %}
**Prerequisites**: Before diving into configuration, review the [Threading Model](/talon/concepts-and-architecture/threading-model) page to understand the architectural concepts and design rationale behind Talon's threading architecture.
{% endhint %}

## Thread Reference

### Talon Microservice Threads

| Thread                               | Name                                                       | Critical Path | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
| ------------------------------------ | ---------------------------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **AEP Engine Input Multiplexer**     | `X-STEMux-<appName>-<instanceid>`                          | **Yes**       | The engine thread that dequeues and dispatches microservice messages and events. This is the main microservice thread on which microservice events are dispatched. They are suffixed with a global counter to allow differentiating between stats emitted by multiple instances of the same microservice running in the same JVM. The detached threads described below can offload work from this thread which can improve throughput and latencies in your microservice. **Note**: When running in an XVM you will see AEP engine threads created for the XVM's admin microservice which is used to handle XVM admin requests. |
|                                      | `X-EventMultiplexer-Wakeup-<appName>`                      | No            | A timer thread used to wake up and dispatch events scheduled via the engine's input queue.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| **Detached Inbound Message Logger**  | `X-ODS-StoreLog-<appName>.in`                              | No            | When the microservice is configured with a detached inbound message logger, this thread offloads the work of writing to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes. As inbound message loggers aren't used for HA purposes they are not on the critical path. **Tip**: If your microservice's inbound message load is not high, a detached inbound message logger may not be needed. The `tleg3` transaction latency statistic covers inbound message logging. High values or spikes are an indicator that a detached inbound message logger can help.                        |
| **Detached Outbound Message Logger** | `X-ODS-StoreLog-<appName>.out`                             | No            | When the microservice is configured with a detached outbound message logger, this thread offloads the work of writing to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes. **Tip**: If your microservice's outbound message load is not high, a detached outbound message logger may not be needed. The `tleg3` transaction latency statistic covers outbound message logging. High values or spikes are an indicator that a detached outbound message logger can help.                                                                                                             |
| **Per Transaction Stats Logger**     | `X-ODS-StoreLog-<appName>.txnstats`                        | No            | When the microservice is configured with a detached per transaction stats logger, this thread offloads the work of writing to disk from the engine multiplexer which can serve as a buffer against disk I/O spikes. **Tip**: If your microservice's transaction load is not high, a detached per transaction stats logger may not be needed. The `cepilo` (commit epilogue) transaction latency statistic covers per transaction stats logging costs. High values or spikes in `cepilo` are an indicator that a detached per transaction stats logger can help.                                                                 |
| **Detached Bus Send Thread**         | `X-AEP-BusManager-IO-<appName>.<busName>`                  | **Yes**       | When the bus is configured for detached send, this thread offloads the work of serialization and writing of outbound messages from the engine's input multiplexer which serves as a buffer against spikes caused by message bus flow control. **Tip**: High values in the `o2p`, `s`, `s2w`, `ws` message bus binding stats are indicators that a detached bus sender can improve performance.                                                                                                                                                                                                                                  |
| **Bus Binding Opener**               | `X-AEP-BusManager-BindingOpener-<appName>.<busName>`       | No            | Each bus configured for your microservice is managed by a Bus Manager internal to the AEP engine. Handles establishment of the bus connections and reconnects.                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| **Store Reader Thread**              | `X-ODS-StoreReplicatorLinkReader-<storeName>-<memberName>` | **Yes**       | The IO thread for the store which is used to read replication traffic from cluster peers.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| **Detached Store Persister**         | `X-ODS-StoreLog-<storeName>-<instanceid>`                  | No            | When the store is configured for detached persistence, this thread offloads the work of writing recovery data to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes. **Tip**: High values in the `pers` (persistence) store binding stats are an indicator that a detached store sender can improve performance.                                                                                                                                                                                                                                                                      |
| **Detached ICR Sender**              | `X-ODS-Store-ICR-Sender-<storeName>-<instanceid>`          | No            | When the store is configured for detached inter-cluster replication, this thread offloads the work of writing recovery data to the receiver from the engine's input multiplexer which can serve as a buffer against disk I/O spikes. **Tip**: High values in the `icr` (inter-cluster replication) store binding stats are an indicator that a detached store sender can improve performance.                                                                                                                                                                                                                                   |
| **Detached Store Send Thread**       | `X-ODS-StoreReplicatorSender-<storeName>-<memberName>`     | **Yes**       | When the store is configured for detached send, this thread offloads the work of writing recovery data to the network for backup instances from the engine's input multiplexer which can serve as a buffer against network I/O spikes. **Tip**: High values in the `s2w` (serialize to wire) store binding stats are an indicator that a detached store sender can improve performance.                                                                                                                                                                                                                                         |
| **Detached Store Dispatch Thread**   | `X-ODS-StoreReplicatorDispatcher-<storeName>-<memberName>` | **Yes**       | When the store is configured for detached dispatch, this thread allows the store reader thread to offload work of dispatching deserialized replication traffic to the engine for processing. This is useful in cases where the cost of deserializing replication traffic is high. **Tip**: A high value for the store binding deserialize stat (`d`) can indicate that setting this property could improve throughput or latency.                                                                                                                                                                                               |
| **Store Acceptor Thread**            | `X-ODS-StoreLinkAcceptor-<instanceid>`                     | No            | Each store configured for clustering will create a thread that will listen for connection requests from other store members. Once the connection is established it is handed off to the store reader thread for processing.                                                                                                                                                                                                                                                                                                                                                                                                     |
| **Stats Printer Threads**            | `X-Stats-Printer [<statName>-<instanceid>.stats]`          | No            | Several components can be configured to trace stats (independently) from XVM collected stats. When such stats threads are enabled a thread is created to periodically print stats. This is typically useful if a microservice is run outside of an XVM.                                                                                                                                                                                                                                                                                                                                                                         |
| **Scheduler**                        | `X-Scheduler-<instance-count>`                             | No            | A timer thread used for scheduling events. An AepEngine uses this to perform periodic engine health checks, for example.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

### Discovery Threads

| Thread              | Name          | Critical Path | Description                                                                                                                                                                                                                                                                                                                                       |
| ------------------- | ------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Discovery Timer** | `X-EDP-Timer` | No            | Each discovery provider that is opened will create a timer thread that will periodically wake up to perform discovery broadcasts. Each discovery provider typically will create additional threads specific to the discovery provider type. For example, when using an SMA-based discovery provider, message bus binding threads will be created. |

### XVM Threads

| Thread                      | Name                                         | Critical Path | Description                                                                                                                                                                                                                                                           |
| --------------------------- | -------------------------------------------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **XVM main thread**         | `X-Server-<xvmName>-Main`                    | No            | This thread creates and starts microservices at startup and upon completion will drive XVM acceptors for accepting admin and direct connections to the XVM.                                                                                                           |
| **XVM stats collector**     | `X-Server-<xvmName>-StatsRunner`             | No            | Collects stats for microservices, and populates them into heartbeats that can be traced, logged, dispatched, and emitted.                                                                                                                                             |
| **XVM dedicated IO thread** | `X-Server-<xvmName>-IOThread-<threadNumber>` | **Yes**       | When the XVM is configured for multithreading, additional IO threads beyond the XVM main thread are created that service connections that are affinitized to it. **Note**: When using the direct binding the IO thread is on the critical path for received messages. |

***

## Critical Thread Affinity Configuration Reference

Threads that can/should be affinitized include the following:

### Engine Input Multiplexer

The engine thread that dequeues and dispatches microservice messages and events. This is the main microservice thread.

The detached threads described below can offload work from this thread which can improve throughput and latencies in your microservice.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <!-- Engine Input Multiplexer -->
    <inboundEventMultiplexing>
      <queueWaitStrategy>BusySpin</queueWaitStrategy>
      <queueDrainerCpuAffinityMask>[2]</queueDrainerCpuAffinityMask>
    </inboundEventMultiplexing>
  </app>
</apps>
```

### Bus Detached Sender Thread

Each bus configured for your microservice can optionally be configured to send committed outbound messages on a detached thread. When the bus is configured for detached send, this thread offloads the work of serialization and writing of outbound messages from the engine's input multiplexer which serves as a buffer against spikes caused by message bus flow control.

**Tip**: High values in the `o2p`, `s`, `s2w`, `ws` message bus binding stats are indicators that a detached bus sender can improve performance.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <!-- Bus Detached Sender Thread -->
    <messaging>
      <buses>
        <bus name="orderprocessing-bus" enabled="true">
          <detachedSend enabled="true">
            <queueWaitStrategy>BusySpin</queueWaitStrategy>
            <queueDrainerCpuAffinityMask>[1]</queueDrainerCpuAffinityMask>
          </detachedSend>
        </bus>
      </buses>
    </messaging>
  </app>
</apps>
```

### Store Reader Thread

The IO thread for the store. On a primary instance, this is the thread that dispatches store acknowledgements back into the engine. On a backup, this is the thread that dispatches received replication traffic.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <storage enabled="true">
      <clustering>
        <linkReaderCpuAffinityMask>[4]</linkReaderCpuAffinityMask>
      </clustering>
    </storage>
  </app>
</apps>
```

### Store Detached Send Thread

When the store is configured for detached send, this thread offloads the work of writing recovery data to the network for backup instances from the engine's input multiplexer which can serve as a buffer against network I/O spikes.

**Tip**: High values in the `s2w` (Serialize To Wire) store binding stats are an indicator that a detached store sender can improve performance.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <storage enabled="true">
      <clustering>
        <detachedSend enabled="true">
          <queueWaitStrategy>BusySpin</queueWaitStrategy>
          <queueDrainerCpuAffinityMask>[5]</queueDrainerCpuAffinityMask>
        </detachedSend>
      </clustering>
    </storage>
  </app>
</apps>
```

### Store Detached Dispatch Thread

When the store is configured for detached dispatch, this thread allows the store reader thread to offload work of dispatching deserialized replication traffic to the engine for processing. This is useful in cases where the cost of deserializing replication traffic is high.

**Tip**: A high value for the store binding deserialize stat (`d`) can indicate that setting this property could improve throughput or latency.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <storage enabled="true">
      <clustering>
        <detachedDispatch enabled="true">
          <queueWaitStrategy>BusySpin</queueWaitStrategy>
          <queueDrainerCpuAffinityMask>[5]</queueDrainerCpuAffinityMask>
        </detachedDispatch>
      </clustering>
    </storage>
  </app>
</apps>
```

### Store Detached Persister

When the store is configured for detached persistence, this thread offloads the work of writing recovery data to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes.

**Tip**: High values in the `pers` (Persistence) store binding stats are an indicator that a detached store sender can improve performance.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <storage enabled="true">
      <clustering>
        <persistence enabled="true">
          <detachedPersist enabled="true">
            <queueWaitStrategy>BusySpin</queueWaitStrategy>
            <queueDrainerCpuAffinityMask>[5]</queueDrainerCpuAffinityMask>
          </detachedPersist>
        </persistence>
      </clustering>
    </storage>
  </app>
</apps>
```

### Store Detached ICR

When the store is configured for detached Inter Cluster Replication, this thread offloads the work of writing recovery data to the ICR bus to insulate the engine's input multiplexer from spikes caused by flow control on the ICR bus.

**Tip**: High values in the `icr` (Inter Cluster Replication) store binding stats are an indicator that a detached store ICR sender can improve performance.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <storage enabled="true">
      <clustering>
        <icr enabled="true">
          <detachedSend enabled="true">
            <queueWaitStrategy>BusySpin</queueWaitStrategy>
            <queueDrainerCpuAffinityMask>[5]</queueDrainerCpuAffinityMask>
          </detachedSend>
        </icr>
      </clustering>
    </storage>
  </app>
</apps>
```

### Detached Inbound Message Logger

When the microservice is configured with a detached inbound message logger, this thread offloads the work of writing to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes.

**Tip**: If your microservice's inbound message load is not high, a detached inbound message logger may not be needed. The `tleg3` transaction latency statistic covers inbound message logging. High values or spikes are an indicator that a detached inbound message logger can help.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <!-- Inbound message logging -->
    <inboundMessageLogging policy="UseDedicated">
      <detachedWrite enabled="true">
        <queueWaitStrategy>BusySpin</queueWaitStrategy>
        <queueDrainerCpuAffinityMask>[7]</queueDrainerCpuAffinityMask>
      </detachedWrite>
    </inboundMessageLogging>
  </app>
</apps>
```

### Detached Outbound Message Logger

When the microservice is configured with a detached outbound message logger, this thread offloads the work of writing to disk from the engine's input multiplexer which can serve as a buffer against disk I/O spikes.

**Tip**: If your microservice's outbound message load is not high, a detached outbound message logger may not be needed. The `tleg3` transaction latency statistic covers outbound message logging. High values or spikes are an indicator that a detached inbound message logger can help.

```xml
<apps>
  <app name="orderprocessing-app" mainClass="com.acme.MyApp">
    <!-- Outbound message logging -->
    <outboundMessageLogging policy="UseDedicated">
      <detachedWrite enabled="true">
        <queueWaitStrategy>BusySpin</queueWaitStrategy>
        <queueDrainerCpuAffinityMask>[6]</queueDrainerCpuAffinityMask>
      </detachedWrite>
    </outboundMessageLogging>
  </app>
</apps>
```

### Bus-Specific Threads

In addition to the core threads above, some bus bindings also support additional threads which may be affinitized.

#### Solace Binding

| Property                                    | Property Name                      | Description                                                                                                                                                                                                                                                                                                                                                                                                                |
| ------------------------------------------- | ---------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Detached Dispatch Thread**                | `dispatcher_cpu_affinity_mask`     | When set to true a detached dispatch thread is created that offloads the work of deserializing messages and dispatching from the solace thread reading messages from the wire. **Tip**: For microservices in which deserialization and dispatch cost is high, enabling detached dispatch can improve throughput and decrease latency.                                                                                      |
| **Solace Consumer Session Dispatch Thread** | `consumer_cpu_affinity_mask`       | This property attempts to affinitize the Solace client library's receiver thread for this connection's consumer session. **Tip**: For highly active sessions the Solace thread can take up a fair amount of CPU and it is on the critical latency path. **Note**: Because this thread is not under the platform's control the thread is affinitized the first time it calls into the binding.                              |
| **Solace Producer Session Dispatch Thread** | `producer_cpu_affinity_mask`       | This property attempts to affinitize the Solace client library's receiver thread for this connection's producer session. **Tip**: Acknowledgements come in on the producer session. This thread is not on the critical latency path, so it doesn't always warrant a core of its own. **Note**: Because this thread is not under the platform's control the thread is affinitized the first time it calls into the binding. |
| **Detached Send Thread**                    | `detached_sends_cpu_affinity_mask` | Configures the CPU affinity mask for detached sender thread. **Tip**: Enabling the detached send thread is usually not needed for Talon microservices because they can use the engine's bus detached send thread described in the previous section. This property may be useful for microservices using the Solace binding outside of a Talon microservice.                                                                |

Example of configuring affinities for Solace bindings:

```xml
<buses>
  <bus name="orderprocessing-bus">
    <provider>solace</provider>
    <address>192.168.1.10</address>
    <port>55555</port>
    <properties>
      <detached_sends>true</detached_sends>
      <detached_sends_cpu_affinity_mask>[1]</detached_sends_cpu_affinity_mask>

      <detached_dispatch>true</detached_dispatch>
      <dispatcher_cpu_affinity_mask>[2]</dispatcher_cpu_affinity_mask>

      <single_session>false</single_session>
      <producer_cpu_affinity_mask>0</producer_cpu_affinity_mask>
      <consumer_cpu_affinity_mask>[9]</consumer_cpu_affinity_mask>
    </properties>
  </bus>
</buses>
```

#### Direct Binding

The direct binding allows microservices to connect directly to the microservice via the XVM. When using the direct binding it is possible to set up a dedicated IOThread for accepting and servicing direct connections. The IOThread can be affinitized using the affinity attribute:

```xml
<buses>
  <bus name="orderprocessing-bus">
    <provider>direct</provider>
    <address>orderprocessing-app</address>

    <!--
      Not a cpu affinity mask, this property instructs clients
      to use a particular io thread configured in the xvm. The
      cpu affinity for the io thread is configured below.
    -->
    <threadaffinity>1</threadaffinity>
  </bus>
</buses>

<servers>
  <server name="orderprocessing-vm" group="default" template="default">
    <acceptors>
      <acceptor descriptor="${ORDERPROCESSING_ACCEPTOR::tcp://0.0.0.0:0}" />
    </acceptors>
    <multiThreading enabled="true">
      <ioThreads>
        <ioThread id="0" affinity="0" enabled="true" />

        <!--
          The affinity property below configures the cpu affinity
          mask for the IO thread handling connections made to it.
        -->
        <ioThread id="1" affinity="[1]" enabled="true" />
      </ioThreads>
    </multiThreading>
  </server>
</servers>
```

## Related Topics

* [Threading Model](/talon/concepts-and-architecture/threading-model) - Architectural concepts and design rationale
* [Disruptors](/talon/developing-applications/configuring-the-runtime/threading/disruptors) - Configure LMAX disruptor ring buffers
* [Thread Affinitization](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) - Pin threads to CPU cores for optimal performance
* [DDL Reference](/talon/reference/configuration) - Complete DDL syntax reference

## Next Steps

1. Identify which threads are in your microservice's critical path
2. Use thread statistics to determine which threads are busy-spinning
3. Review [Thread Affinitization](/talon/developing-applications/configuring-the-runtime/threading/thread-affinitization) for approaches to pinning threads
4. Configure affinity masks for critical threads


# Thread Affinitization

Pin threads to specific CPU cores for optimal performance, reduced jitter, and NUMA optimization.

{% hint style="info" %}
**Prerequisites**: Before diving into configuration, review the [Threading Model](/talon/concepts-and-architecture/threading-model) page to understand the architectural concepts and design rationale behind Talon's threading architecture.
{% endhint %}

## Overview

To achieve the lowest possible latency and best throughput with minimal jitter, Talon supports the ability to pin critical threads to individual CPU cores. This section presents two approaches for affinitizing your microservice.

## Thread Affinitization

To achieve the lowest possible latency and best throughput with minimal jitter, Talon supports the ability to pin critical threads to individual CPU cores. This section presents two approaches for affinitizing your microservice.

### Terminology and Concepts

Before diving into configuration details, let's review key terminology:

* **CPU Socket**: Refers to a physical connector on a motherboard that accepts a single physical chip. Modern CPUs provide multiple physical cores which are exposed to the operating system as logical CPUs that can perform parallel execution streams. [See Also: CPU Socket](https://en.wikipedia.org/wiki/CPU_socket)
* **NUMA**: **N**on-**U**niform **M**emory **A**ccess, refers to the commonplace architecture in which machines with multiple CPU sockets divide the memory banks of RAM into nodes on a per-socket basis. Access to memory on a socket's "local" memory node is faster than accessing memory on a remote node tied to a different socket. [See Also: NUMA](https://en.wikipedia.org/wiki/Non-uniform_memory_access)
* **CPU Core**: Contemporary CPUs are likely to run multiple cores which are exposed to the underlying OS as a CPU. [See Also: Multi-core processing](https://en.wikipedia.org/wiki/Multi-core_processor)
* **Hyper-threading**: Intel technology to make a single core appear logically as multiple cores on the same chip to improve performance
* **Logical CPU**: What the operating system sees as a CPU. The number of CPUs available to the OS is: `<num sockets> * <cores per socket> * <hyper threads per core>`
* **Processor Affinity**: Refers to the act of restricting the set of logical CPUs on which a particular program thread can execute

### Benefits of Thread Affinitization

Pinning a thread to a particular CPU ensures that the OS won't reschedule the thread to another core and incur a context switch that would force the thread to reload its working state from main memory, which results in jitter. When all critical threads in the processing pipeline are pinned to their own CPU and busy spinning, the OS scheduler is less likely to schedule another thread onto that core, keeping the threads' processor caches hot.

### Preparing for Affinitization

To get the most out of affinitization, each busy-spinning thread should be pinned to its own CPU core which prevents the operating system from relocating the thread to another logical CPU while the program is executing.

#### Identifying Busy-Spinning Threads

Any platform threads that are marked as critical in the [Thread Reference](/talon/developing-applications/configuring-the-runtime/threading/thread-reference) page should be affinitized. An easy way to see what threads are busy spinning is to enable XVM thread stats and trace:

```
ID    CPU       DCPU    DUSER   CPU%  USER% STATE           NAME
28    1.1s      2.5ms   2.5ms   1     101   RUNNABLE        X-Server-myapp-1-StatsRunner
46    14.5m     5.2s    5.2s    100   100   RUNNABLE        X-STEMux-myapp-2
47    14.5m     5.2s    5.2s    100   101   RUNNABLE        X-AEP-BusManager-IO-myapp.my-bus
....
```

Assuming you have enough CPUs on your machine such that two critical threads aren't scheduled on the same CPU, any thread that is consistently using >90% CPU while your microservice is not processing messages is one that will benefit from affinitization. Determining the number of busy-spinning threads will allow you to determine if it is possible to pin them all to processors on the same NUMA node.

#### Determining CPU Layout

CPU layout is machine dependent. Before configuring CPU affinity masks, it is necessary to determine the CPU layout on the target machine. Talon includes a utility class, `UtlThread`, that can be run to assist with this:

```bash
java -cp "libs/*" com.neeve.util.UtlThread
```

which will produce output similar to the following:

```
0: CpuInfo{socketId=0, coreId=0, threadId=0}
1: CpuInfo{socketId=1, coreId=0, threadId=0}
2: CpuInfo{socketId=0, coreId=8, threadId=0}
3: CpuInfo{socketId=1, coreId=8, threadId=0}
4: CpuInfo{socketId=0, coreId=2, threadId=0}
5: CpuInfo{socketId=1, coreId=2, threadId=0}
6: CpuInfo{socketId=0, coreId=10, threadId=0}
7: CpuInfo{socketId=1, coreId=10, threadId=0}
8: CpuInfo{socketId=0, coreId=1, threadId=0}
9: CpuInfo{socketId=1, coreId=1, threadId=0}
10: CpuInfo{socketId=0, coreId=9, threadId=0}
11: CpuInfo{socketId=1, coreId=9, threadId=0}
12: CpuInfo{socketId=0, coreId=0, threadId=1}
13: CpuInfo{socketId=1, coreId=0, threadId=1}
14: CpuInfo{socketId=0, coreId=8, threadId=1}
15: CpuInfo{socketId=1, coreId=8, threadId=1}
16: CpuInfo{socketId=0, coreId=2, threadId=1}
17: CpuInfo{socketId=1, coreId=2, threadId=1}
18: CpuInfo{socketId=0, coreId=10, threadId=1}
19: CpuInfo{socketId=1, coreId=10, threadId=1}
20: CpuInfo{socketId=0, coreId=1, threadId=1}
21: CpuInfo{socketId=1, coreId=1, threadId=1}
22: CpuInfo{socketId=0, coreId=9, threadId=1}
23: CpuInfo{socketId=1, coreId=9, threadId=1}
```

In the above, we can see:

* The machine has 24 logical CPUs (0 through 23)
* There are two processor sockets (socketId=0, socketId=1)
* There are 12 physical cores total - 6 physical cores per socket (coreIds 0, 1, 2, 8, 9, and 10)
* Hyper-threading is enabled and there are two threads per socket (threadId=0, threadId=1)

{% hint style="info" %}
**Note**: The fashion in which the OS assigns core numbers is OS dependent.

**Linux Only**: The UtlThread class is only supported on Linux currently. Eventually, support for other platforms will be added.
{% endhint %}

#### Best Practices

* Before launching your process, validate that there aren't other processes running that are spinning on a core to which you are affinitizing
* Check what other processes on the host will use busy spinning and find out the cores they will use
* In Linux, the OS often uses Core 0 for some of its tasks, so it is better to avoid this core if possible
* When feasible it is best to disable hyper-threading to maximize the amount of CPU cache available to each CPU

### Approach 1: Basic Affinitization

The basic affinitization approach requires no additional DDL configuration and simply uses the `numactl` command to restrict the NUMA memory nodes and logical CPUs on which your microservice can execute. Using this approach can be a good first step in evaluating the performance benefits of affinitizing your microservice, but is not ideal for reducing jitter.

**Pros:**

* Simple
* Avoids remote NUMA node access

**Cons:**

* Does not prevent thread context switches; the OS is free to move threads between logical CPUs which leads to jitter
* Does not provide visibility into what CPU a thread is running on, making it harder to diagnose cases where 2 critical threads are scheduled on the same core

#### Launching with Basic Affinitization

In the CPU layout determined above, one could launch a microservice with memory pinned to NUMA node 1, and only CPUs from socket 1 as follows:

```bash
numactl -m1 -C1,3,5,7,9,11 java -cp "libs/*" com.neeve.server.Main -n orderprocessing-vm
```

Refer to your [`numactl` manual pages](https://linux.die.net/man/8/numactl) for more information.

#### Validating Basic Affinitization

With basic affinitization, it isn't straightforward to determine what CPUs any particular thread ends up running on, but you can use a command like `top` to validate that all of your microservice threads are running on the expected nodes, by pressing the '1' key after launching top. With enough effort, it may be possible to correlate the thread IDs displayed in a stack dump to those shown in a tool such as htop, but that is outside the scope of this document.

### Approach 2: Advanced Affinitization

For microservices that are most concerned with reducing jitter, the basic affinitization approach described above still leaves open the potential for the operating system relocating your threads from one CPU to another which can lead to latency spikes. With the advanced affinitization approach described here, you will avoid this by configuring each busy-spinning or critical thread in the microservice to its own CPU to avoid context switching.

#### CPU Affinity Mask Format

Thread affinities are configured by supplying a mask that indicates the cores on which a thread can run. The mask can either be a long bit mask of logical CPUs, or a square bracket enclosed comma-separated list enumerating the logical CPUs to which a thread should be affinitized. The latter format is recommended as it is easier to read.

Examples:

* `"0"` - no affinity specified (0x0000)
* `"[]"` - no affinity specified
* `"1"` - specifies logical CPU 0 (0x0001)
* `"[0]"` - specifies logical CPU 0
* `"4"` - specifies logical CPU 2 (0x0100)
* `"[2]"` - list specifying logical CPU 2
* `"6"` - mask specifying logical CPU 1 and 2 (0x0110)
* `"4294967296"` - specifies logical CPU 32 (0x1000 0000 0000 0000 0000 0000 0000 0000)
* `"[32]"` - specifies logical CPU 32
* `"[1,2]"` - list specifying logical CPU 1 and 2

#### Enabling Affinitization

By default, CPU affinitization is disabled. To enable it you can set the following env flags in the DDL configuration:

```xml
<env>
  <nv>
    <enablecpuaffinitymasks>true</enablecpuaffinitymasks>
    <defaultcpuaffinitymask>[0]</defaultcpuaffinitymask>
  </nv>
</env>
```

#### Configuring CPU Affinities

**Step 1: Configure Default CPU Affinity Mask**

Threads that are critical for reducing microservice latency and improving throughput are listed in the reference tables above, but not all threads are critical. To prevent non-critical threads from being scheduled on a CPU being used by a critical thread, the platform allows the microservice to configure one or more 'default' CPUs that non-critical threads can be affinitized to, by setting the `nv.defaultcpuaffinitymask` environment variable. For example, the platform's statistics collection thread doesn't need its own dedicated CPU to perform its relatively simple tasks of periodically reporting heartbeats. However, we still want to ensure that the operating system doesn't try to schedule it onto the same core as a critical thread, so the platform will affinitize it with the default CPU affinity mask.

**Step 2: Configure Critical Platform Threads Affinities**

Critical platform-related threads are those that have the most impact on latency and performance. When the platform is optimized for latency or throughput these threads will be set to use BusySpin or Yielding respectively to avoid being context switched. Each of these threads should be assigned its own CPU.

See the [Critical Thread Affinity Configuration Reference](/talon/developing-applications/configuring-the-runtime/threading/thread-reference#critical-thread-affinity-configuration-reference) section below for a listing of these threads and how to configure their affinities.

**Step 3: Affinitizing Non-Platform Threads**

If your microservice uses its own threads, they can be affinitized as well by using the platform's `UtlThread` utility class. Non-critical threads that are not busy-spinning threads should be affinitized to the default cores and critical or busy threads should be pinned to their own core to prevent them from being scheduled on top of the platform's threads.

**Non-Critical, Non-Spinning Thread**

Non-critical threads can be affinitized to the set of default CPUs configured by `nv.defaultcpuaffinitymask` by calling `setDefaultCpuAffinityMask` from the thread to be affinitized:

```java
com.neeve.util.UtlThread.setDefaultCpuAffinityMask();
```

**Critical or Busy Threads**

Threads that participate in your transaction's processing flow or are spinning or heavy CPU users should be pinned to their own core so that they don't interfere with affinitized platform threads. For example:

```java
com.neeve.util.UtlThread.setCpuAffinityMask("[9]");
```

#### Launching with NUMA Affinitization

Unlike with the Basic Affinitization approach, when all threads have been affinitized to their own core or the default core, it is not strictly necessary to restrict what cores a process operates on, just the memory node to which to restrict the process. In fact, it can even be beneficial to let threads outside the platform or microservice's control be scheduled on other NUMA nodes.

```bash
numactl -m0 java -cp "libs/*" com.neeve.server.Main -n orderprocessing-vm
```

#### Validating Affinitization

**Via Thread Stats Output**

The easiest way to check your work is to enable XVM thread stats. Thread stats are emitted in heartbeats and affinities can be reported in monitoring tools. If the XVM is configured to trace thread stats, then thread usage is printed as follows:

```
ID    CPU       DCPU    DUSER   CPU%  USER% STATE           NAME
28    1.1s      2.5ms   2.5ms   1     101   RUNNABLE        X-Server-myapp-1-StatsRunner (aff=[1(s0c1t0)])
46    14.5m     5.2s    5.2s    100   100   RUNNABLE        X-STEMux-myapp-2 (aff=[6(s0c9t0)])
47    14.5m     5.2s    5.2s    100   101   RUNNABLE        X-AEP-BusManager-IO-myapp.my-bus (aff=[4(s0c4t0)])
```

You can look for any spinning thread (CPU% at 100) that doesn't have an affinity assigned. This will help you avoid the following pitfalls:

* Having two threads spinning on the same coreId will make performance worse (either same coreId but different threadId or worse on the same coreId/threadId)
* Having some other non-Talon process spinning on one of the coreIds that you've affinitized to
* Affinitizing across multiple socketIds (which are on different NUMA nodes) can make performance worse
* You will be limited in your max heap to the amount of physical memory in that processor bank of the NUMA node to which you are pinning

The platform outputs thread affinitization using the format like: `(aff=[6(s0c9t0)])` which can be interpreted as logical CPU 6 which is on **s**ocket 0, **c**ore 9, **t**hread 0.

**Programmatically**

```java
UtlThread.dumpAffinitizationState(System.out, "  ");
```

This will dump the affinitization state of all threads affinitized through UtlThread.

**Via Trace**

The above trace will also be printed by an AepEngine after messaging has been started or alternatively when it assumes a backup role (in most cases all platform threads will have been started by this time).

#### Limitations

The following limitations apply to thread affinitization support:

* Thread affinitization is currently only supported on Linux
* Affinitization is limited to being able to affinitize threads to logical cores 0 through 63
* Affinitization of a thread does not reserve the CPU core, just limits the cores on which a thread will execute. This is important because if not all threads are affinitized the OS thread scheduler may schedule another thread on top of a critical thread if CPU resources are scarce

***

## Related Topics

* [Threading Model](/talon/concepts-and-architecture/threading-model) - Architectural concepts and design rationale
* [Thread Reference](/talon/developing-applications/configuring-the-runtime/threading/thread-reference) - Complete reference of all Talon threads and their configuration
* [Disruptors](/talon/developing-applications/configuring-the-runtime/threading/disruptors) - Configure LMAX disruptor ring buffers
* [DDL Reference](/talon/reference/configuration) - Complete DDL syntax reference

## Next Steps

1. Review the [Threading Model](/talon/concepts-and-architecture/threading-model) to understand NUMA and affinitization
2. Determine your machine's CPU layout using `UtlThread`
3. Start with basic affinitization to evaluate benefits
4. Move to advanced affinitization if jitter reduction is critical
5. Test performance with and without affinitization


# Discovery

Configure discovery providers for cluster formation, XVM discovery, and administrative tool connectivity.

## Overview

Discovery configuration determines how Talon components find and communicate with each other. Proper discovery configuration is essential for:

* **Application Clustering**: Enabling application instances to find peers and form HA clusters
* **Administrative Tools**: Allowing tools to discover and manage running XVMs

## Topics

* [**Discovery Configuration**](/talon/developing-applications/configuring-the-runtime/discovery/discovery-configuration) - Configure discovery providers and descriptors

## Related Topics

* [Discovery Model](/talon/concepts-and-architecture/discovery-model) - Understanding discovery concepts and architecture
* [Admin Over SMA](/talon/developing-applications/configuring-the-runtime/administration/admin-over-sma) - Using SMA-based discovery for administration
* [Discovery Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/discovery-tool) - Troubleshooting discovery issues


# Discovery Configuration

Configure discovery providers, cache properties, and separate discovery domains for XVM and cluster discovery.

## Overview

Discovery caches are configured using descriptor strings that specify the provider, address, and additional properties. All discovery cache implementations support common properties for controlling entity lifecycle and advertisement behavior.

## Discovery Descriptor Format

A discovery cache descriptor follows this format:

```
<provider>://<address>[&prop1=value]...[&propN=valueN]
```

For example:

```
mcast://224.0.1.200:4090&initWaitTime=10&maxEntityAge=30
```

## Common Discovery Properties

The following properties are supported by all discovery cache implementations:

| Property       | Default | Description                                                                                                                                                                                                                                                                                                                                         |
| -------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `initWaitTime` | 5       | <p>Time (in seconds) to wait after sending solicit packets for entities to join.<br><br>Upon construction, cache members solicit entity advertisements from remote members to populate the local cache. The initial wait time is the time that the cache constructor will sleep waiting for remote entities to be populated in the local cache.</p> |
| `maxEntityAge` | 30      | <p>Time (in seconds) that can elapse without receiving an ESA before an entity is deemed dead.<br><br>The maximum age associated with each entity created by the local cache member. This parameter works hand in hand with the <code>maxEsaLoss</code> parameter to determine how often to transmit ESAs.</p>                                      |
| `maxEsaLoss`   | 6       | <p>Number of ESAs, if lost, that would cause an entity to be expired (i.e., number of ESAs to advertise within the maxEntityAge interval).<br><br>An entity owner determines how often to broadcast ESAs using the maxEntityAge and maxESALoss parameters: ESAs are broadcast every <code>maxEntityAge / maxESALoss</code> seconds.</p>             |
| `memberName`   | UUID    | <p>Each discovery cache member is uniquely identified by a name. The member name is used to identify entity ownership.<br><br>By default, each member is uniquely identified by a UUID. However, you can override the use of UUIDs for identification through the use of the memberName property.</p>                                               |

## The Default Discovery Cache

Talon broadcasts Applications, XVMs, and ODS Stores via discovery. Unless configured otherwise, these will be done using the default (global) discovery cache singleton in the JVM.

The default discovery descriptor uses IP multicast and defaults to `mcast://224.0.1.200:4090`.

This can be changed by configuring the environment property `nv.discovery.descriptor`:

**DDL Configuration:**

```xml
<env>
  <nv>
    <discovery>
      <descriptor>mcast://224.0.1.200:4096</descriptor>
    </discovery>
  </nv>
</env>
```

**System Property:**

```bash
-Dnv.discovery.descriptor=mcast://224.0.1.200:4096
```

{% hint style="info" %}
**Tip**: If you are on the same network as several of your peers and you are all working on the same application, it would be a good idea to choose your own port for discovery to avoid clashing.
{% endhint %}

## Configuring Separate XVM and Cluster Discovery

Using the default discovery provider for both XVM and cluster discovery is sufficient for most applications. However, there may be cases where it is advantageous to use separate providers.

**Example:** Using SMA for cluster discovery while using multicast for XVM discovery:

```xml
<env>
  <nv>
    <discovery>
      <descriptor>mcast://224.0.1.200:4096</descriptor>
    </discovery>
  </nv>
</env>

<apps>
  <app name="MyApp">
    <storage>
      <clustering>
        <!-- Using discovery element -->
        <discovery>
          <provider>solace</provider>
          <address>192.168.1.100</address>
          <port>55555</port>
          <properties>
            <discoveryChannel>cluster-discovery</discoveryChannel>
          </properties>
        </discovery>

        <!-- OR using discoveryDescriptor -->
        <discoveryDescriptor>solace://192.168.1.101:55555&amp;discoveryChannel=cluster-discovery</discoveryDescriptor>
      </clustering>
    </storage>
  </app>
</apps>

<xvms>
  <xvm name="MyXVM">
    <!-- Using discovery element -->
    <discovery>
      <provider>solace</provider>
      <address>192.168.1.101</address>
      <port>55555</port>
      <properties>
        <discoveryChannel>xvm-discovery</discoveryChannel>
      </properties>
    </discovery>

    <!-- OR using discoveryDescriptor -->
    <discoveryDescriptor>solace://192.168.1.101:55555&amp;discoveryChannel=xvm-discovery</discoveryDescriptor>
  </xvm>
</xvms>
```

## Provider-Specific Configuration

### Multicast Discovery

The multicast provider is the default discovery mechanism.

**Descriptor Format:**

```
mcast://<multicast-address>:<port>[&properties]
```

**Multicast-Specific Properties:**

| Property      | Default | Description                                                                                                                                                             |
| ------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `localIfAddr` | N/A     | <p>When running on a host with multiple network interfaces that support multicast, this property specifies the interface that should be used for discovery.<br><br></p> |

{% hint style="info" %}
\
\*\*Tip\*\*: A common issue with multicast discovery occurs when 2 hosts bind to adapters that are on different discovery networks.<br>
{% endhint %}

|

**Example:**

```xml
<env>
  <nv>
    <discovery>
      <descriptor>mcast://224.0.1.200:4090&amp;localIfAddr=192.168.1.10</descriptor>
    </discovery>
  </nv>
</env>
```

{% hint style="warning" %}
**Use IPv4**: Multicast should be run over an IPv4 stack. When running on an OS or JVM that prefers IPv6, you should explicitly specify the usage of IPv4 by setting `-Djava.net.preferIPv4Stack=true`
{% endhint %}

{% hint style="warning" %}
**Mac OS X Yosemite or Later**: The Yosemite OS X version and beyond create a virtual interface (awdl0) for Airplay that states it has multicast enabled, but multicast does not work if that interface is discovered by the multicast provider. There is a tool called [WiFried](https://medium.com/@mariociabarra/wifriedx-in-depth-look-at-yosemite-wifi-and-awdl-airdrop-41a93eb22e48) that allows one to selectively disable this interface temporarily.
{% endhint %}

### SMA Discovery

The SMA discovery provider uses topic broadcasts for advertising packets. Any broker-based SMA messaging provider can be used for discovery.

**Descriptor Format:**

```
<provider>://<address>:<port>[&properties]
```

**SMA-Specific Properties:**

| Property           | Default | Description                                                                                                                                                                   |
| ------------------ | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `discoveryChannel` | *XEDP*  | The channel name to create and use for discovery. Because message bus topics start with the channel name by default, this value is used as the topic name on the message bus. |

Any bus binding-specific properties can be passed in with the discovery descriptor. The bus binding used for discovery will use the `memberName` as the value passed to the bus binding provider's create method.

{% hint style="warning" %}
**Resilience Consideration**: When considering using SMA-based discovery, you might want to consider using a different messaging server than the one used for application traffic, particularly if you will be using monitoring tools that rely on XVM discovery to discover the hosts that they manage. A failure in the messaging fabric would otherwise leave applications undiscoverable.
{% endhint %}

**Example SMA Discovery Descriptors:**

**Solace:**

When using the Solace provider, consider using the `single_session=true` property to reduce the number of threads created by the Solace binding. The Solace bindings created by the SMA discovery provider are marked as administrative and as such platform autotuning will not result in busy-spinning threads.

```xml
<env>
  <nv>
    <discovery>
      <descriptor>solace://solacehost:55555&amp;single_session=true</descriptor>
    </discovery>
  </nv>
</env>
```

**Loopback:**

The loopback provider uses the loopback bus and can be used for discovery within the same process. It can be useful in testing scenarios to simulate a message-based discovery provider, but it is often more efficient to use the local discovery provider described below.

```xml
<env>
  <nv>
    <discovery>
      <descriptor>loopback://xvm-discovery</descriptor>
    </discovery>
  </nv>
</env>
```

**JMS:**

Configuring a generic JMS discovery provider uses JNDI:

```xml
<xvm>
  <discovery>
    <provider>jms</provider>
    <address>tibcohost</address>
    <port>27326</port>
    <properties>
      <username>admin</username>
      <password>changeme</password>
      <jndi>true</jndi>
      <jndi_contextfactory>com.tibco.tibjms.naming.TibjmsInitialContextFactory</jndi_contextfactory>
      <jndi_principal>admin</jndi_principal>
      <jndi_credentials>changeme</jndi_credentials>
      <jndi_connectionfactory>CSTopicConnectionFactory</jndi_connectionfactory>
      <maxEntityAge>150</maxEntityAge>
      <maxEsaLoss>30</maxEsaLoss>
      <discoveryChannel>=_XEDP_/dev</discoveryChannel>
    </properties>
  </discovery>
</xvm>
```

**ActiveMQ:**

```xml
<env>
  <nv>
    <discovery>
      <descriptor>activemq://brokerhost:61616</descriptor>
    </discovery>
  </nv>
</env>
```

**Kafka:**

```xml
<env>
  <nv>
    <discovery>
      <descriptor>kafka://kafka-broker:9092&amp;discoveryChannel=talon-discovery</descriptor>
    </discovery>
  </nv>
</env>
```

### Local Discovery

The local discovery provider is a simple provider that can be used to find other entities in the same process. It is similar to the SMA loopback provider but has lower overhead.

**Descriptor Format:**

```
local://.
```

**Use Cases:**

* Unit tests launching all applications in the same JVM
* Development with collocated applications
* Testing scenarios requiring isolated discovery

**Example:**

```xml
<env>
  <nv>
    <discovery>
      <descriptor>local://.</descriptor>
    </discovery>
  </nv>
</env>
```

## Related Topics

* [Discovery Model](/talon/concepts-and-architecture/discovery-model) - Understanding discovery concepts
* [Admin Over SMA](/talon/developing-applications/configuring-the-runtime/administration/admin-over-sma) - Administrative discovery configuration
* [Discovery Tool](/talon/operating-applications/analysis-and-troubleshooting/tools/discovery-tool) - Troubleshooting discovery

## Next Steps

1. Choose appropriate discovery provider for your environment
2. Configure default discovery cache for XVM discovery
3. Optionally configure separate cluster discovery for applications
4. Test discovery configuration using the Discovery Tool
5. Monitor entity advertisements and age-out behavior


# Operating Applications

This section covers the operational aspects of running Talon applications in production, organized into four key areas: Deployment, Administration, Monitoring, and Analysis & Troubleshooting.

## Overview

Operating Talon applications involves:

* **Administration**: Service launch, shutdown, and command-and-control operations
* **Monitoring**: Receiving and viewing service telemetry (statistics)
* **Deployment**: Preparing the environment a service starts in
* **Analysis & Troubleshooting**: Logging and tools for analyzing logs and telemetry

## Operational Components

### [Administration](/talon/operating-applications/administration)

Tools and techniques for managing Talon services:

* [**Admin Tool**](/talon/operating-applications/administration/admin-tool) - Command-line administrative interface
* [**Admin Over SMA**](/talon/operating-applications/administration/admin-over-sma) - Remote administration via messaging

### [Monitoring](/talon/operating-applications/monitoring)

Real-time telemetry and statistics collection:

* [**XVM Heartbeats**](/talon/operating-applications/monitoring/xvm-heartbeats) - Server-level statistics and health monitoring
* [**Engine Stats**](/talon/operating-applications/monitoring/engine-statistics) - Message processing and engine metrics
* [**Per Transaction Stats**](/talon/operating-applications/monitoring/per-transaction-statistics) - Transaction-level performance metrics

### [Deployment](/talon/operating-applications/deployment)

Preparing the environment a service starts in:

* [**Native Libraries**](/talon/operating-applications/deployment/native-libraries) - Extraction of the platform's bundled native libraries

### [Analysis & Troubleshooting](/talon/operating-applications/analysis-and-troubleshooting)

Logging and analysis tools for troubleshooting:

* [**Trace Logging**](/talon/operating-applications/analysis-and-troubleshooting/trace-logging) - Enable and configure trace logging
* [**The Transaction Log Tool**](/talon/operating-applications/analysis-and-troubleshooting/tools/transaction-log-tool) - Work with transaction logs
* [**Querying Transaction Logs**](/talon/operating-applications/analysis-and-troubleshooting/querying-transaction-logs) - Query binary transaction logs
* [**Stats Dump Tool**](/talon/operating-applications/analysis-and-troubleshooting/tools/stats-dump-tool) - Offline analysis of heartbeat logs

## Related Topics

* [Operating Model](/talon/concepts-and-architecture/operating-model) - Conceptual overview of operating Talon applications
* [Configuring Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - How to configure statistics

## Next Steps

1. Start with [Administration](/talon/operating-applications/administration) to learn how to manage running applications
2. Configure [Monitoring](/talon/operating-applications/monitoring) to collect statistics and health metrics
3. Set up [Trace Logging](/talon/operating-applications/analysis-and-troubleshooting/trace-logging) for troubleshooting


# Administration

This section covers tools and techniques for administering Talon applications, including service launch, shutdown, and command-and-control operations.

## Overview

Talon provides multiple administrative interfaces for managing running applications:

* Command-line tools for direct administration
* Remote administration via messaging infrastructure

## Administrative Tools

* [**Admin Tool**](/talon/operating-applications/administration/admin-tool) - Command-line administrative interface
* [**Admin Over SMA**](/talon/operating-applications/administration/admin-over-sma) - Remote administration via messaging

## Related Topics

* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Define custom administrative commands
* [Monitoring](/talon/operating-applications/monitoring) - View application telemetry
* [Analysis & Troubleshooting](/talon/operating-applications/analysis-and-troubleshooting) - Analyze logs and troubleshoot issues


# Admin Tool

## Overview

The AdminTool is a lightweight tool that provides the ability for invoking commands on a running XVM or Application. It is intended primarily for use by developers; dedicated administrative tools are recommended for production use cases. The AdminTool discovers running XVMs and connects to them over TCP via their advertised XVM [acceptor](/talon/reference/configuration).

## Running The Admin Tool

The AdminTool can be found in the nvx-talon or nvx-core-all jar. And can be invoked as follows:

```bash
java -Dnv.discovery.descriptor=<xvm-discovery-descriptor> -cp "libs\*" com.neeve.tools.AdminTool

__   __      _____  _            _______ ______ ____  _____  __  __
\ \ / /     |  __ \| |        /\|__   __|  ____/ __ \|  __ \|  \/  |
 \ V /      | |__) | |       /  \  | |  | |__ | |  | | |__) | \  / |
  > <       |  ___/| |      / /\ \ | |  |  __|| |  | |  _  /| |\/| |
 / . \      | |    | |____ / ____ \| |  | |   | |__| | | \ \| |  | |
/_/ \_\     |_|    |______/_/    \_\_|  |_|    \____/|_|  \_\_|  |_|
XVM Admin Tool                             nvx-talon v3.12 (build 7)
                               Copyright(c) 2019 Neeve Research, LLC
                                                 All Rights Reserved
Type 'help' for the list of commands
admin>
```

... where the libs folder contains the talon jar and its dependencies. The same discovery descriptor used by the XVM must be specified when launching the tool as the AdminTool discovers running XVMs and connects to them via the [acceptor](/talon/reference/configuration) that each XVM has broadcast.

## Listing Running XVMs

The 'xvms' command can be used to see which XVMs have been discovered:

```bash
admin> xvms
XVM...
...cardholdermaster-1-1 [host = MY-LAPTOP, connected= false]
...cardholdermaster-2-1 [host = MY-LAPTOP, connected= false]
...cardmaster-1-1 [host = MY-LAPTOP, connected= false]
...cardmaster-2-1 [host = MY-LAPTOP, connected= false]
...fraudanalyzer-1-1 [host = MY-LAPTOP, connected= false]
...fraudanalyzer-2-1 [host = MY-LAPTOP, connected= false]
...merchantmaster-1-1 [host = MY-LAPTOP, connected= false]
...merchantmaster-2-1 [host = MY-LAPTOP, connected= false]
...perfdriver-1 [host = MY-LAPTOP, connected= false]
```

## List Apps in an XVM

The list of apps loaded in an XVM can be determined by using the 'admin' which invokes a command on the XVM that returns the list of loaded apps.

```bash
admin> admin perfdriver-1 app_list
Invoking 'app_list' --> xvm 'perfdriver-1'...
OK[admin,perfdriver]
admin>
```

## List App Commands

The list of apps loaded in an XVM can be determined using the listAppCommand command as follows. The following shows an example of listing the commands for the perfdriver app shown to be running in the perfdriver-1 XVM in the example above.

```bash
admin> listAppCommands -x perfdriver-1 -a perfdriver -u
Fetching commands for app='perfdriver' in xvm 'perfdriver-1'...
Found '11 commands:

...

getAuthorizationResponseCount ("Get Authorization Response Count")
  Gets the number of authorizations received
  Usage:
  getAuthorizationResponseCount
getAuthorizationRequestCount ("Get Authorization Request Count")
  Gets the number of authorizations requested
  Usage:
  getAuthorizationRequestCount
stopAuthorizationRequests ("Stop Sending")
  Halts Requests being sent to driven app
  Usage:
  stopAuthorizationRequests
sendAuthorizationRequests ("Send Authorization Requests")
  Drives Authorization Request traffic
  Usage:
  sendAuthorizationRequests [-c] [-r] [-a]
       [<-c|--count> The rate at which to send requests default='10000']
       [<-r|--rate> The rate at which to send requests default='1000']
       [<-a|--async> Whether or not to spin up a background thread to do
           the sends default='true']
seedMerchants ("Seed Merchants")
  Seeds merchants with their stores.
  Usage:
  seedMerchants [-c] [-r]
       [<-c|--count> The number of merchants to seed default='100']
       [<-r|--rate> The rate at which to send in merchants default='100']
seedCardHolders ("Seed Card Holders")
  Seeds card holders with their transaction history.
  Usage:
  seedCardHolders [-c] [-r] [-a]
       [<-c|--count> The number of card holders to seed default='100']
       [<-r|--rate> The rate at which to send in card holders
           default='100']
       [<-a|--async> Whether or not to spin up a background thread to do
           the sends default='true']
```

## Invoking an App Command

A command on an application can be invoked using the invoke command. The following shows an example of invoking the getAuthorizationRequestCount listed in the section above.

```bash
admin> invoke perfdriver perfdriver-1 getAuthorizationRequestCount
Invoking 'getAuthorizationRequestCount' --> app='perfdriver' in xvm 'perfdriver-1'...
OK[1000]
admin>
```

## Command Reference

### Tool Properties

The following properties can be set using `set <Property Name> <Property Value>`:

| Property Name                 | Default Value | Description                                                                                                                                                               |
| ----------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `commandTimeout`              | 30.0s         | Sets the timeout to wait for a command response. If no unit suffix is specified the value is interpreted as SECONDS                                                       |
| `connectHandshakeTimeout`     | 1.5s          | The connect timeout. If no unit suffix is specified the value is interpreted as SECONDS                                                                                   |
| `longCommandDisplayThreshold` | 5.0s          | Commands that take longer than the given time will result in the tool outputting their execution time. If no unit suffix is specified the value is interpreted as SECONDS |
| `stacktraces`                 | false         | Whether or not command exception stacktraces should be dumped                                                                                                             |

### Commands

#### listXvms | xvms

Shows available XVMs

```
Usage:
  listXvms [-h]

       [<-h|--help> Displays this help message (default='false')]
```

#### disconnect

Closes currently open admin connections

```
Usage:
  disconnect [-h] [<xvm>]

       [<-h|--help> Displays this help message (default='false')]
       [xvm: The XVM in whose connection should be closed (if omitted
           closes all connections) ]
```

#### connect

Connects to an XVM

```
Usage:
  connect [-h] <xvm>

       [<-h|--help> Displays this help message (default='false')]
       xvm: The XVM to connect to.
```

#### listAppCommands

List commands for an XVM or an app hosted by a XVM

```
Usage:
  listAppCommands [-h] -x [-a] [-u] [<filter>]

       [<-h|--help> Displays this help message (default='false')]
       <-x|--xvm> The XVM in which to list commands
       [<-a|--app> The app whose command to list. If 'admin' or omitted
           then XVM administration commands are listed. (default='admin')]
       [<-u|--usage> Flag that can be specified to additionally show usage
           for the commands. (default='false')]
       [filter: Optionally can be specified to list only commands that
           contain this filter in their name. '*' indicates that all commands
           should be displayed default='*']
```

#### invoke

Sends a command to a XVM or an app hosted by a XVM

```
Usage:
  invoke [-h] [<app>] <xvm> <command>

       [<-h|--help> Displays this help message (default='false')]
       [app: The app whose command to list. If 'xvm' or omitted then XVM
           commands are listed. default='admin']
       xvm: The XVM in which to list commands
       command: The name of the command to execute
       [args: The command arguments]
```

#### admin

Sends a command to a XVM or an app hosted by a XVM

```
Usage:
  admin [-h] <dest> <command>

       [<-h|--help> Displays this help message (default='false')]
       dest: The app and XVM against which to invoke the command specified
           as <appName>@<xvmName>.
 The appName can be omitted if the command
           is an XVM command.
       command: The name of the command to execute
       [args: The command arguments]
```

### General Commands

#### get

Gets a configuration or environment property

```
Usage:
  get [-h] [-a] [<propName>]

       [<-h|--help> Displays this help message (default='false')]
       [<-a|--all> flag indicating that all properties should be listed
           (default='false')]
       [propName: The name of property to get, or with -a, a filter on
           properties to list]
```

#### reset

Reset a configuration or environment property to its default value

```
Usage:
  reset [-h] <propName>

       [<-h|--help> Displays this help message (default='false')]
       propName: The name of property to reset. '*' resets all properties
           to their default value.
```

#### set

Sets a configuration or environment property

```
Usage:
  set [-h] [<propName>] [<propValue>]

       [<-h|--help> Displays this help message (default='false')]
       [propName: The name of property to set, if no name is set config
           properties are listed]
       [propValue: The value of the property to set. If omitted clears the
           property]
```

#### stacktraces

Sets whether or not command exception stacktraces should be shown

```
Usage:
  stacktraces [-h] <enabled>

       [<-h|--help> Displays this help message (default='false')]
       enabled: <true|false|off|on> Indicates whether or not command
           exceptions should be displayed
```

#### history

Displays command history

```
Usage:
  history [-h] [-c] [<n>]

       [<-h|--help> Displays this help message (default='false')]
       [<-c|--clear> Clears command history]
       [n: lists only the last 'n' lines default='1000']
```

#### help

Displays help message

```
Usage:
  help [-h] [<command>]

       [<-h|--help> Displays this help message (default='false')]
       [command: When specifies displays help for that command only]
```

#### ansi

Enables or disables ansi output

```
Usage:
  ansi [-h] [<value>]

       [<-h|--help> Displays this help message (default='false')]
       [value: <off|on> Turns ansi on or off. With no argument display the
           current ansi setting]
```

#### echo

Displays a message or turns echo on or off

```
Usage:
  echo [-h] [<value>]

       [<-h|--help> Displays this help message (default='false')]
       [value: Displays a message or turns echo on or off. With no argument
           list the current echo setting]
       [message: A message to display default='']
```

#### script

Runs a command script

```
Usage:
  script [-h] [<script>]

       [<-h|--help> Displays this help message (default='false')]
       [script: The script file to execute]
```

#### bye | exit | quit

Exit the tool.

```
Usage:
  bye [-h]

       [<-h|--help> Displays this help message (default='false')]
```

## Related Topics

* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Define custom commands for your applications
* [Admin Over SMA](/talon/operating-applications/administration/admin-over-sma) - Remote administration via messaging
* [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats) - Monitor XVM statistics

## Next Steps

1. Configure discovery to match your XVM deployment
2. Launch Admin Tool and discover running XVMs
3. Connect to target XVM
4. List applications and their available commands
5. Invoke commands interactively or via scripts


# Admin Over SMA

{% hint style="info" %}
**Since 3.10**
{% endhint %}

## Overview

In addition to the standard direct (TCP based) monitoring capabilities, a Talon XVM can be configured to allow administrative applications to monitor and manage it over messaging. When Admin over SMA is enabled, XVMs emit heartbeat, trace and lifecycle events over defined messaging channels for consumption by listening clients. Administrative and monitoring tools use discovery to discover available XVMs. Once an Admin Client discovers an XVM it connects to the configured message bus and subscribes to its monitoring channels. At a high level, all that is required to enable Admin over SMA is to set the following properties:

```properties
nv.discovery.descriptor=<discovery-provider>
nv.server.admin.transports=sma
nv.server.admin.bus.descriptor=<bus-connection-descriptor>
```

As long as both the admin client and the XVM are using the same discovery descriptor and message bus, they will be able to communicate with one another.

{% hint style="warning" %}
**Warning**: When enabling Admin over SMA it is important to note that by default admin clients using SMA will attempt to connect to discovered XVMs. If older version XVMs or XVMs not configured for Admin over SMA are advertising themselves over the admin client's discovery address the XVM will never respond and the admin client will see connection timeouts. To avoid this you may:

* Use [passive monitoring](#passive-monitoring) on the client side to avoid attempts to ping the XVM
* Use separate XVM discovery for XVMs that support Admin over SMA vs. those that don't
  {% endhint %}

## Admin Channels

The table below lists the bus channels used for administration over SMA:

| Name                    | Key                                   | QOS        | ID    | Description                                             |
| ----------------------- | ------------------------------------- | ---------- | ----- | ------------------------------------------------------- |
| **Command and Control** |                                       |            |       |                                                         |
| xvm-request             | *xvm-admin/${xvmName}/request*        | BestEffort | 30000 | Command and control requests                            |
| xvm-response            | *xvm-admin/${adminClientId}/response* | BestEffort | 30001 | Command and control responses (to issuing admin client) |
| **Monitoring**          |                                       |            |       |                                                         |
| xvm-heartbeat           | *xvm-admin/${xvmName}/heartbeat*      | BestEffort | 30002 | XVM heartbeats                                          |
| xvm-trace               | *xvm-admin/${xvmName}/trace*          | BestEffort | 30003 | Log trace records                                       |
| xvm-event               | *xvm-admin/${xvmName}/event*          | BestEffort | 30004 | Lifecycle and alert events                              |

By default, all admin channels start with *xvm-admin* channel prefix. This prefix can be changed by setting the environment property `nv.admin.sma.channelKeyPrefix` which can be useful for cases where it is desirable to more granularly partition admin traffic. Note that each channel is configured with a topic level of either *xvmName* or *adminClientId*. The *xvmName* topic level allows clients to listen in to a particular XVM, and the *adminClientId* allows an XVM to send responses targeted to a particular admin client.

### Request and Response Channels

When command and control is enabled, these channels allow admin clients to issue commands to an XVM. Such commands include XVM control commands such as triggering thread dumps or reloading applications and also include invocation of application-defined commands.

### Heartbeats

XVM heartbeats are periodically emitted over the heartbeats channel when they are [enabled for the XVM](/talon/operating-applications/monitoring/xvm-heartbeats#configuring-heartbeats).

Heartbeats are emitted using the [SrvMonHeartbeatMessage](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/SrvMonHeartbeatMessage.html) type defined in the [com.neeve.server.mon](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/package-summary.html) package. While this package contains additional lifecycle related messages types used by direct monitoring clients, only [SrvMonHeartbeatMessage](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/SrvMonHeartbeatMessage.html) are emitted over the heartbeats channel.

### Trace

When enabled, SrvMonTraceRecords are emitted over the trace channel allowing admin clients to view trace output as it happens.

Trace records are emitted using [SrvMonTraceRecord](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/cnc/ISrvMonTraceRecord.html) message type that can be found in the [com.neeve.server.mon.cnc](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/cnc/package-summary.html) package.

### Events

The events channel is used to emit events such as lifecycle or alert events which are defined in the [com.neeve.server.mon.lifecycle](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/lifecycle/package-summary.html) and [com.neeve.server.mon.alert](https://build.neeveresearch.com/core/javadoc/LATEST/com/neeve/server/mon/alert/package-summary.html) packages.

## Configuration

### Discovery

Admin clients should not attempt to issue commands until an XVM has been discovered via the discovery provider (see [Understanding Discovery](/talon/developing-applications/configuring-the-runtime/discovery)). When an admin client detects that an XVM is no longer discoverable it should stop issuing commands over SMA.

### Configuring Admin Client Connectivity

Administrative tools, such as the AdminTool, must be configured to enable SMA as a transport along with the admin bus connection information. As admin clients won't always be configured via DDL, configuration is done via system/environment properties to configure connectivity to the XVMs monitoring server.

**Properties Based Configuration with Bus Descriptor:**

```properties
nv.server.admin.transports=sma
nv.server.admin.sma.bus.descriptor=solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true
```

Alternatively, it is possible to configure the bus descriptor in decomposed form, which can be useful in cases where configuration properties are more overridden across environments.

**Properties Based Configuration for Clients:**

```properties
nv.server.admin.transports=sma
nv.server.admin.sma.bus.provider=solace
nv.server.admin.sma.bus.address=solhost
nv.server.admin.sma.bus.port=55555
nv.server.admin.sma.bus.properties.SESSION_VPN_NAME=default
nv.server.admin.sma.bus.properties.use_default_queue_name=false
nv.server.admin.sma.bus.properties.use_default_queue_name_as_default_client_id
nv.server.admin.sma.bus.properties.topic_starts_with_channel=false
nv.server.admin.sma.bus.properties.usejni=true
nv.server.admin.sma.bus.properties.single_session=true
```

### Configuring XVMs

XVMs can be configured using the same environment properties as clients by setting the properties in the DDL environment section:

```xml
<env>
  <nv>
    <server>
      <admin>
        <transports>sma</transports>
        <sma>
          <bus>
            <descriptor>solace://solhost:55555&topic_starts_with_channel=false&SESSION_VPN_NAME=default&use_default_queue_name=false&use_default_queue_name_as_default_client_id=true&topic_starts_with_channel=false&usejni=true&single_session=true</descriptor>
          </bus>
        </sma>
      </admin>
    </server>
  </nv>
</env>
```

#### Advanced XVM Bus Configuration

Using environment based configuration is the simplest way of configuring Admin over SMA for an XVM. In cases where bus configuration is not being injected by deployment tools, it is possible to use the `<xvm>` `<admin>` element in DDL to enable admin over SMA and reference a DDL defined bus definition.

**XVM Admin over SMA Configuration**

To configure an XVM to use a bus named 'xvm-admin' the XVM's `<admin>` `<sma>` element can be used:

```xml
<xvms>
  <xvm name="order-processing-1" template="xvm-template">
    <admin>
      <transports>
        <sma enabled="true">
          <busName>xvm-admin</busName>
        </sma>
      </transports>
    </admin>
    <heartbeats enabled="true" interval="5s"/>
  </xvm>
</xvms>
```

**Admin Bus Configuration**

The following bus definition can then be configured for use by the XVM.

```xml
<buses>
  <bus name="xvm-admin">
    <provider>solace</provider>
    <address>solhost</address>
    <port>55555</port>
  </bus>
</buses>
```

Admin channels (xvm-request, xvm-response, xvm-heartbeat, xvm-event, and xvm-trace) should not be configured for the bus, they are automatically created by the XVM when it is started.

With the above configuration, the `order-processing-1` XVM will create a connection to solace://solhost:55555 with a username of `order-processing-1`. It will use the following topics:

**Subscribe:**

* **xvm-request channel:** xvm-admin/order-processing-1/request

**Publish:**

* **xvm-response channel:** xvm-admin/${adminClientId}/response (where the adminClientId is substituted with that of the sending client when a response is sent)
* **xvm-heartbeats channel:** xvm-admin/order-processing-1/heartbeat (when heartbeats are enabled)
* **xvm-trace channel:** xvm-admin/order-processing-1/trace (when trace emission is enabled)
* **xvm-event channel:** xvm-admin/order-processing-1/event

{% hint style="info" %}
**Tip**: Enabling XVM heartbeats isn't strictly necessary, but in most monitoring scenarios it is desirable.
{% endhint %}

### Passive Monitoring

It is possible to use Admin over SMA in a purely passive monitoring capacity by setting the property:

```properties
nv.server.admin.passivemonitoringonly=true
```

With the above configuration setting clients will throw an exception if an attempt is made to send commands to an XVM, and XVMs will not issue subscriptions on the xvm-request channel.

## Related Topics

* [XVM Heartbeats](/talon/operating-applications/monitoring/xvm-heartbeats) - Configure heartbeat emission
* [Admin Tool](/talon/operating-applications/administration/admin-tool) - Command-line administrative interface
* [Implementing Command Handlers](/talon/developing-applications/authoring-user-code/command-and-control/implementing-command-handlers) - Define custom commands

## Next Steps

1. Configure discovery provider for XVM and admin clients
2. Set up message bus for admin channels
3. Enable Admin Over SMA in XVM configuration
4. Configure admin tools to use SMA transport
5. Test admin operations via messaging


# Monitoring

This section covers receiving and viewing telemetry (statistics) from running Talon applications.

## Overview

Talon automatically collects and reports statistics at multiple levels:

* **XVM (Server) Statistics**: Load average, memory usage, per-thread CPU utilization
* **Engine Stats**: Message throughput, object pool usage, transaction statistics
* **Per-Transaction Statistics**: Detailed performance metrics at the transaction level

By default, statistics reporting is conservative to minimize overhead. You can configure the level of detail based on your monitoring needs.

## Monitoring Components

* [**XVM Heartbeats**](/talon/operating-applications/monitoring/xvm-heartbeats) - Server-level statistics and health monitoring
* [**Memory Stats**](/talon/operating-applications/monitoring/memory-statistics) - Detailed memory statistics (heap, off-heap, IO buffers, entity lifecycle)
* [**Engine Stats**](/talon/operating-applications/monitoring/engine-statistics) - Message processing and engine metrics reference
* [**Per Transaction Stats**](/talon/operating-applications/monitoring/per-transaction-statistics) - Transaction-level performance metrics

## Related Topics

* [Exposing Application Stats](/talon/developing-applications/authoring-user-code/monitoring/exposing-application-statistics) - Define custom telemetry in your microservices
* [Configuring Monitoring](/talon/developing-applications/configuring-the-runtime/monitoring) - Configure statistics collection
* [Administration](/talon/operating-applications/administration) - Manage running applications
* [Analysis & Troubleshooting](/talon/operating-applications/analysis-and-troubleshooting) - Analyze collected telemetry




---

[Next Page](/llms-full.txt/1)