Separate hot and cold fields in JqObject for better cache-line utilization

[stalep]

Summary

Move JqObject's lazily-computed cache fields behind a single indirection to keep the hot data path (keys, values, size) compact and cache-line friendly. This follows the "cold data separation" principle from Google's Abseil Performance Hints:

Place hot read-only fields away from hot mutable fields so that writes to the mutable fields do not cause the read-only fields to be evicted from nearby caches. Move cold data so it does not live next to hot data, either by placing the cold data at the end of the struct, or behind a level of indirection.

Background

Current JqObject layout

public final class JqObject implements JqValue {
    // Hot fields (accessed on every get/has/forEach/appendTo):
    private final String[] keys;          // 8 bytes (reference)
    private final JqValue[] values;       // 8 bytes (reference)
    private final int size;               // 4 bytes
    private final Map<String, JqValue> externalMap;  // 8 bytes (reference, usually null)

    // Cold fields (accessed only on specific operations):
    private transient String[] sortedKeysCache;      // 8 bytes — only for compareTo/keys builtin
    private transient JqArray cachedKeysArray;        // 8 bytes — only for keys builtin
    private transient Map<String, JqValue> mapView;   // 8 bytes — only for objectValue()
    private transient int[] hashSlots;                // 8 bytes — only for objects >32 keys
    private transient int hashMask;                   // 4 bytes — only for objects >32 keys
}

Total object size: 16 bytes (header) + 68 bytes (fields) = ~84 bytes. With alignment padding, this spans 2+ cache lines (64 bytes each).

The problem

When iterating an array of JqObjects (e.g., [.pcp_time_series[] | .["mem.util.used"]]), the CPU loads each JqObject into cache. The hot fields (keys, values, size) are needed for get(), but the cold fields (sortedKeysCache, cachedKeysArray, mapView) are loaded into the same cache lines even though they're not accessed. This wastes cache line capacity and increases L1 miss rate.

perf stat data (2026-07-02) confirms the impact:

• prod_extractMetric: 4,658 L1-dcache-load-misses (5.69% miss rate)
• Iterates 502 JqObjects, accessing only keys[], values[], hashSlots[] per object

Proposed change

Move cold fields behind a lazily-allocated cache object:

public final class JqObject implements JqValue {
    // Hot fields only:
    private final String[] keys;
    private final JqValue[] values;
    private final int size;
    private final Map<String, JqValue> externalMap;

    // Single reference to cold data (null until first use):
    private transient volatile LazyCache cache;

    private static class LazyCache {
        String[] sortedKeysCache;
        JqArray cachedKeysArray;
        Map<String, JqValue> mapView;
        int[] hashSlots;
        int hashMask;
    }
}

New object size: 16 bytes (header) + 36 bytes (4 refs + 1 int + padding) ≈ 48-56 bytes — fits in a single cache line.

The LazyCache is allocated on first access to any cold method (sortedKeys(), sortedKeysAsArray(), objectValue(), hashLookup() for >32 keys). For the hot path (get() on ≤32 key objects), the cache is never allocated and never touched.

Impact on hash index

For objects with >32 keys, hashSlots[] and hashMask move to LazyCache. The get() method for large objects would need one extra pointer dereference to reach the hash index. However:

• hashSlots[] is built eagerly at construction time for >32 key objects (via ofArrays()). The LazyCache would be allocated at construction time for these objects — no lazy allocation overhead on the hot path.
• The extra indirection is one L1 cache hit (the LazyCache reference), which is 0.5ns — negligible compared to the hash lookup itself.

What does NOT change

• get(String key) for ≤32 key objects: still linear scan on keys[]/values[], no LazyCache access
• has(String key): same as get()
• forEach(BiConsumer): iterates keys[]/values[] directly, no LazyCache access
• appendTo(StringBuilder): iterates arrays directly
• size(): reads size field directly
• entries(), keys(), values(): these access objectValue() which goes through LazyCache — but these are already "cold" operations

Interaction with other issues

• Share key arrays across JqObjects with identical schemas to reduce heap pressure and cache misses #48 (shared key arrays): compatible — the shared keys[] reference is still a hot field
• Investigate columnar (structure-of-arrays) representation for arrays of uniform JqObjects #49 (columnar arrays): compatible — columnar layout is about the JqArray level, not JqObject internals
• Prepare for Project Valhalla value classes (JDK 28+) #54 (Valhalla value classes): JqObject stays as an identity class regardless, so this optimization remains valid
• Reduce INTERN_TABLE from 2048 to 512 slots to improve L1 cache locality during parsing #47 (smaller intern table): independent — intern table is in JqValues, not JqObject

Benchmark Plan

java -jar jjq-benchmark/target/jjq-benchmark-*.jar \
  JjqProductionQueryBenchmark.prod_extractMetric \
  JjqProductionQueryBenchmark.prod_iterateExtract \
  JjqProductionQueryBenchmark.prod_objectConstruct \
  -f 3 -wi 5 -i 5 -prof perfnorm

Compare before/after:

• L1-dcache-load-misses per operation (expect reduction on extractMetric)
• Throughput on iteration-heavy benchmarks
• No regression on single-object access (prod_topField, prod_deepField)

Acceptance Criteria

• Cold fields moved behind LazyCache indirection
• get() / has() / forEach() / appendTo() do not access LazyCache for ≤32 key objects
• Hash index built into LazyCache at construction time for >32 key objects
• All existing tests pass (478+ in jjq-core, 991 in test-suite)
• Measurable L1-dcache-load-miss reduction on prod_extractMetric
• No regression on single-object access benchmarks
• Serialization (Serializable support from JqValue should implement Serializable for Hibernate second-level cache support #34) works correctly with LazyCache as transient

References

• Abseil Performance Hints — "Move cold data behind a level of indirection"
• perf stat data from 2026-07-02 profiling session
• jjq Reduce INTERN_TABLE from 2048 to 512 slots to improve L1 cache locality during parsing #47: Smaller intern table for L1 cache locality
• jjq Share key arrays across JqObjects with identical schemas to reduce heap pressure and cache misses #48: Shared key arrays across uniform JqObjects
• jjq Investigate columnar (structure-of-arrays) representation for arrays of uniform JqObjects #49: Columnar arrays for uniform JqObjects
• jjq Prepare for Project Valhalla value classes (JDK 28+) #54: Valhalla value classes preparation