Package com.helix.core.ml

Class OnnxSessionPool

java.lang.Object

com.helix.core.ml.OnnxSessionPool

All Implemented Interfaces:

AutoCloseable

public class OnnxSessionPool
extends Object
implements AutoCloseable

Thread-safe, high-throughput runtime session pool for in-process ONNX model inference.

Manages bounded pools of native OrtSession instances per registered model, optimizing multi-threaded virtual-thread execution with zero lock contention, zero JNI leaks, and atomic session draining upon dynamic model reload.

Constructor Summary

Constructors

Constructor	Description
OnnxSessionPool(ModelRegistry modelRegistry)
OnnxSessionPool(ModelRegistry modelRegistry, int maxSessionsPerModel)
OnnxSessionPool(ModelRegistry modelRegistry, OnnxFeatureExtractor featureExtractor, int maxSessionsPerModel)

Method Summary

Modifier and Type	Method	Description
void	close()
OnnxInferenceResult	executeInference(String modelName, ExecutionContext context)	Executes single-row in-process inference for the specified model and context.
OnnxInferenceResult	executeInference(String modelName, String outputTensorName, int outputIndex, ExecutionContext context)	Executes single-row inference with explicit output tensor and index overrides.
float[][]	executeTensorInference(String modelName, String inputTensorName, float[][] inputMatrix, String outputTensorName)	Executes tensor-based inference with raw 2D float matrix inputs and outputs.
void	invalidate(String modelName)	Drains and invalidates cached sessions for a model.
void	reloadModel(String modelName)	Atomically reloads sessions for the named model, draining existing sessions.
static float	run(ExecutionContext context, String modelName)	Static runtime bridge invoked by compiled ASM bytecode.
static float	run(ExecutionContext context, String modelName, String outputTensorName, int outputIndex)	Static runtime bridge invoked by compiled ASM bytecode with explicit output tensor and index.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

OnnxSessionPool

public OnnxSessionPool(ModelRegistry modelRegistry)

OnnxSessionPool

public OnnxSessionPool(ModelRegistry modelRegistry,
 int maxSessionsPerModel)

OnnxSessionPool

public OnnxSessionPool(ModelRegistry modelRegistry,
 OnnxFeatureExtractor featureExtractor,
 int maxSessionsPerModel)

Method Details

run

public static float run(ExecutionContext context,
 String modelName)

Static runtime bridge invoked by compiled ASM bytecode.

Parameters:

context - runtime execution context containing input features

modelName - name of registered ONNX model

Returns:

predicted probability score as primitive float

run

public static float run(ExecutionContext context,
 String modelName,
 String outputTensorName,
 int outputIndex)

Static runtime bridge invoked by compiled ASM bytecode with explicit output tensor and index.

Parameters:

context - runtime execution context containing input features

modelName - name of registered ONNX model

outputTensorName - target output tensor name

outputIndex - class or output index

Returns:

predicted probability score as primitive float

executeInference

public OnnxInferenceResult executeInference(String modelName,
 ExecutionContext context)

Executes single-row in-process inference for the specified model and context.

Parameters:

modelName - name of the active registered model

context - runtime execution context containing required input features

Returns:

populated inference result containing score, predicted class, and latency

executeInference

public OnnxInferenceResult executeInference(String modelName,
 String outputTensorName,
 int outputIndex,
 ExecutionContext context)

Executes single-row inference with explicit output tensor and index overrides.

Parameters:

modelName - model name

outputTensorName - target output tensor name

outputIndex - class or output index

context - evaluation context

Returns:

inference result

executeTensorInference

public float[][] executeTensorInference(String modelName,
 String inputTensorName,
 float[][] inputMatrix,
 String outputTensorName)

Executes tensor-based inference with raw 2D float matrix inputs and outputs.

Parameters:

modelName - name of registered ONNX model

inputTensorName - optional input tensor name (or null to use model default)

inputMatrix - 2D float matrix of inputs

outputTensorName - optional output tensor name (or null to use model default)

Returns:

2D float matrix of output logits

reloadModel

public void reloadModel(String modelName)

Atomically reloads sessions for the named model, draining existing sessions.

Parameters:

modelName - name of model to reload

invalidate

public void invalidate(String modelName)

Drains and invalidates cached sessions for a model.

Parameters:

modelName - name of model to invalidate

close

public void close()

Specified by:

close in interface AutoCloseable