Understanding Message Serialization

Most Talon applications never think about how a message becomes bytes. You declare a message in a model, call sendMessage(), and a handler on another microservice receives the same message back as a Java object. The bus binding does the encoding on the way out and the decoding on the way in.

This page is for the cases where that is not enough:

• You are integrating an application that does not run on Talon, and it has to produce or consume Talon messages directly.

• You are looking at bytes in a wire sniffer, a broker's message browser, or a log, and you need to know what they mean.

• You are writing a custom bus binding, or a tool that reads a message log.

Serialization and Deserialization

Talon messages are ordinary Java objects. A message declared in an ADM model generates a class implementing MessageView, and that interface carries the full serialization surface.

Two things travel on the wire for every message:

• The message payload, the encoded fields of the message itself.

• The message metadata, a small fixed-layout header that tells the receiver how to interpret the payload, and which channel the message was sent on.

The payload alone is not enough to reconstruct a message. The receiver needs the metadata to know which encoding was used and which generated factory and type to hand the bytes to. How the two are carried is a property of the binding: the Solace binding puts the payload in the message body and the metadata in an x-sma-metadata property; the JMS binding does the same over a BytesMessage.

Engine-Independent Serializers

com.neeve.sma.MessageView exposes serialization that does not require an engine, a binding, or any running Talon infrastructure. Given a message object you can turn it into bytes, and given bytes you can turn them back into a message.

The methods come in families, one per destination type:

Destination	Serialize	Deserialize
byte[]	serializeToByteArray(), serializeTo(byte[], int)	deserializeFromByteArray(byte[]), deserializeFrom(byte[], int, int)
ByteBuffer	serializeToByteBuffer(), serializeTo(ByteBuffer)	deserializeFromByteBuffer(ByteBuffer), deserializeFrom(ByteBuffer)
IOBuffer	serializeToIOBuffer(boolean), serializeTo(IOBuffer, int)	deserializeFrom(IOBuffer, int, int)
IOElasticBuffer	serializeTo(IOElasticBuffer, int)	deserializeFrom(IOElasticBuffer, int, int)
PktPacket	serializeToPacket(), serializeTo(PktPacket)	deserializeFromPacket(PktPacket), deserializeFrom(PktPacket)
Native address	serializeTo(long, int)	deserializeFrom(long, int, int)
JSON	serializeToJson()	deserializeFromJson(String)

Note the offset-into-existing-buffer overloads, serializeTo(byte[] array, int offset) and friends. These write into a buffer you already own rather than allocating a new one, and they return the number of bytes written. If you are assembling a larger frame, or you are on a path where allocation matters, these are the ones to reach for; the serializeToXxx() forms allocate.

serializeToJson() and deserializeFromJson() are a convenience for debugging and interoperability. They are not the same thing as the JSON encoding type below, which is the wire encoding negotiated for a channel.

See the MessageView Javadoc for exact signatures and per-method semantics.

Message Metadata

MessageMetadata is the header that accompanies every message. It has a fixed binary layout, which is what makes it readable by an application that is not running Talon.

Message Encoding Type

The encoding type is a single byte identifying how the payload was encoded:

Value	Constant	Encoding
3	ENCODING_TYPE_XBUF	Xbuf
4	ENCODING_TYPE_PROTOBUF	Protobuf
5	ENCODING_TYPE_JSON	JSON
7	ENCODING_TYPE_XBUF2	Xbuf2

There is no value 2.

The legacy Confluence page listed 3 = Protobuf and 4 = Xbuf. Those two are the wrong way round, and the page predates Xbuf2 entirely. The table above is taken from the ENCODING_TYPE_* constants in MessageView. If you have code or notes derived from the old page, check this field.

For new applications, prefer Protobuf, Xbuf2, JSON, or Custom. Xbuf is effectively superseded by Xbuf2. See Choosing an Encoding Type.

Field Reference

Field	Meaning
Version	Metadata wire format version (1 or 2). Determines the layout of everything after it.
Encoding type	How the payload is encoded. See the table above.
View factory	Id of the ADM-generated factory that can create the message.
View type	Id of the message type within that factory. V2 only. On V1 metadata this reads as 0.
Sender	Id of the sending member. 0 when unspecified.
Flow	Message flow id, used for ordering and duplicate detection. 0 when unspecified.
Sno	Message sequence number within the flow. 0 for an unsequenced message.
Channel id	Id of the channel the message was sent on, or a non-positive value if no id was sent.
Channel name length	Length in bytes of the channel name that follows, or -1 if no name was sent.
Channel name	The channel name, if present. Variable length.

The view factory and view type together identify the message class. This is why a V1 receiver has to introspect the payload to work out what it is holding, while a V2 receiver can dispatch straight from the metadata.

Channel id and channel name are the inputs to inbound channel resolution, and a message does not necessarily carry both. See Inbound Channel Resolution for what the receiver does with them, and why a channel id of -1 is normal rather than an error.

V1 Wire Layout

Fixed portion is 24 bytes, followed by the variable-length channel name.

Offset	Size	Field
0	1	Version (1)
1	1	Encoding type
2	2	View factory
4	4	Sender
8	4	Flow
12	8	Sno
20	2	Channel id
22	2	Channel name length (-1 if absent)
24	n	Channel name

V2 Wire Layout

V2 inserts the view type after the view factory, shifting everything below it by two bytes. The fixed portion is 26 bytes.

Offset	Size	Field
0	1	Version (2)
1	1	Encoding type
2	2	View factory
4	2	View type
6	4	Sender
10	4	Flow
14	8	Sno
22	2	Channel id
24	2	Channel name length (-1 if absent)
26	n	Channel name

These offsets correspond to the V1_MESSAGE_*_POS and V2_MESSAGE_*_POS constants in MessageMetadata, and the fixed lengths to FIXED_WIRE_LENGTH_V1 and FIXED_WIRE_LENGTH_V2.

Read the version byte at offset 0 first and branch on it. Do not assume V2: which version a binding emits is configurable, and V1 is still what you get by default on some bindings. On the Solace binding, sma_metadata_version=2 selects V2, see Solace Binding.

Sending Messages from External Applications

An application that does not run on Talon can still exchange messages with one. It has to do by hand what the binding would otherwise do: serialize the payload, build the metadata, and put both where the binding expects to find them.

The mechanics are per-binding, and each binding's page carries a worked example:

• Solace Binding: Sending and Receiving from External Applications, using x-sma-metadata on a BytesMessage.

• JMS Binding: Sending and Receiving from External Applications.

The field semantics those examples rely on are the ones documented above.

Related Topics

• Messaging Model - channels, keys, and inbound channel resolution

• Choosing an Encoding Type

• Sending Messages