Storage Module

The Storage module benchmarks X Platform's ODS (Object Data Store) performance, measuring object operations in a transactional context.

Overview

The ODS provides:

• In-memory object storage

• Transactional semantics

• Indexed access

• Persistence integration

This benchmark measures the overhead of object operations within the X Platform engine.

Test Programs

ESMember

Class: com.neeve.perf.ods.ESMember

Benchmarks object store operations using Event Sourcing HA policy.

Operations Benchmarked

The benchmark measures:

• Object creation

• Object updates

• Object queries (by key)

• Object iteration

• Index lookups

Running Benchmarks

Basic Store Operations Test

$JAVA_HOME/bin/java -cp "libs/*" com.neeve.perf.ods.ESMember \
  --objectCount 1000000 \
  --operationCount 10000000 \
  --operationMix "create:20,update:60,query:20"

Key Parameters

objectCount

Number of objects in the store:

--objectCount 1000000  # 1M objects

Larger object counts test:

• Memory efficiency

• Index performance

• GC impact

operationMix

Ratio of different operations:

--operationMix "create:10,update:70,query:20"  # Typical workload

indexCount

Number of indexes to maintain:

--indexCount 3  # Create 3 indexes

More indexes provide faster queries but slower updates.

Interpreting Results

Typical Results (Linux x86-64)

Operation	Hot Cache	Cold Cache
Object Create	~500ns	~2µs
Object Update	~300ns	~1.5µs
Object Query (indexed)	~200ns	~1µs
Object Iteration	~50ns/object	~100ns/object

Performance Characteristics

1. Hot Cache Performance:

   • Sub-microsecond operations

   • Minimal GC impact

   • Excellent for high-frequency access

2. Memory Efficiency:

   • Compact object representation

   • Efficient index structures

   • Zero-copy where possible

3. Transactional Overhead:

   • Minimal overhead within transactions

   • Batch operations for efficiency

Comparison with AEP Module

The AEP Module includes store operations:

• Storage Module: Measures pure object operation overhead

• AEP Module: Store operations within message processing

• AEP Impact: Store operations add < 1µs to end-to-end latency

Store operations are a tiny fraction of end-to-end latency

Best Practices

For High Performance

1. Use Indexes Wisely:

   • Index frequently-queried fields

   • Avoid over-indexing (impacts updates)

   • Choose appropriate index types

2. Batch Operations:

   • Group related updates in single transaction

   • Reduces transaction overhead

3. Memory Management:

   • Size heap appropriately for object count

   • Monitor GC behavior

   • Use off-heap storage for very large datasets

For Large Datasets

// GOOD: Batch updates in single transaction
transaction(() -> {
    for (Order order : orders) {
        orderStore.update(order);
    }
});

// BAD: Separate transaction per update
for (Order order : orders) {
    transaction(() -> {
        orderStore.update(order);  // Transaction overhead per update!
    });
}

Object Store Features

Indexing

Automatic index maintenance:

@Store
@Indexes({
    @Index(fields = {"customerId"}),
    @Index(fields = {"orderDate", "customerId"})
})
public interface Order {
    String getOrderId();
    String getCustomerId();
    LocalDate getOrderDate();
    // ...
}

Transactions

ACID semantics:

• Atomicity: All or nothing

• Consistency: Constraints enforced

• Isolation: Serializable transactions

• Durability: Persisted if enabled

Replication

Automatic state replication in clustered deployments:

• State changes replicated to backup

• Backup maintains identical state

• Transparent failover

Next Steps

• Review AEP Module to see store usage in end-to-end context

• Review Persistence Module for storage layer benchmarks

• Return to Modules Overview for other modules