Skip to main content

Abstraction Costs & Memory Overhead

Every abstraction in a high-throughput runtime introduces trade-offs between CPU execution speed and GC memory allocation.

Executor Subsystem Memory Costs​

+-----------------------------------------------------------------------------------------+
| Executor Type              | Allocation / Invocation  | Latency Overhead | Peak Throughput        |
+-----------------------------------------------------------------------------------------+
| SyncExecutor               | 0 Bytes (Zero Allocation)| ~8 ns            | 125,000 ops/sec / core |
| VirtualThreadRuleExecutor  | ~48 Bytes (Loom frame)   | ~350 ns          | 950,000+ ops/sec       |
| AsyncExecutor              | ~184 Bytes (Future+Task) | ~1.2 μs          | 85,000 ops/sec         |
| BatchExecutor              | ~64 Bytes (Chunk pointer)| ~2.4 μs          | 450,000 ops/sec        |
+-----------------------------------------------------------------------------------------+

Why SyncExecutor is Zero-Allocation​

  • Evaluates CompiledRule::eval(ExecutionContext) directly on the caller thread stack.
  • Variables and intermediate comparison results reside exclusively on the JVM operand stack.
  • Zero heap objects created; zero pressure on Young Generation GC eden space.

Virtual Threads (Project Loom) Memory Dynamics​

  • Stack Chunk Allocation: Conventional platform threads allocate an entire 1 MB OS thread stack regardless of invocation depth. In contrast, Java 21 Virtual Threads utilize dynamically-sized continuation stack chunks starting at just a few hundred bytes on the heap.
  • High Concurrency Threshold: Helix can easily fan out evaluation across 50,000+ concurrent virtual threads within StructuredTaskScope without risking java.lang.OutOfMemoryError: unable to create native thread.
  • Carrier Thread Unmounting: During any blocking context retrieval, virtual threads unmount from their underlying ForkJoinPool carrier thread, freeing hardware cores for active CPU computation.

Tiered Cache Memory Footprint​

Based on Java Object Layout (JOL) measurements under -XX:+UseCompressedOops:

  • Tier 1 (Strong Caffeine Cache): 1,248 Bytes per rule entry (Key 184B + Bytecode Class reference 640B + Caffeine node metadata 424B).
  • Tier 2 (SoftReference Cache): 1,280 Bytes per rule entry (SoftReference object wrapper 32B + entry node).
  • Tier 3 (WeakReference Cache): 1,272 Bytes per rule entry (WeakReference object wrapper 24B + entry node).