decision model on the neural engine (measured) :3

[huyedits]

Follow-up to #55.

Measured with macmon, which reads the power rails without sudo. Each phase repeats one operation for 6 s.

operation	ANE	CPU	GPU
idle	0.007 W	~0.8 W	~0.07 W
OCR, .accurate, pinned to the ANE	0.52 W	5.2 W	0.07 W
OCR, .fast (CPU-only control)	0.002 W	6.2 W	0.11 W
NLContextualEmbedding (the old decision-model input)	0.000 W	9.6 W	0.06 W
MiniLM Core ML encoder (new)	1.84 W	3.3 W	0.06 W
Vision saliency + classify	0.69 W	5.1 W	0.11 W
Apple Intelligence decide	4.67 W	1.8 W	0.07 W

Apple's contextual embedding runs on the CPU, not the Neural Engine. The decision model now uses all-MiniLM-L6-v2, converted to an fp16 Core ML program: 0.76 ms per sentence, against 8 ms for the old embedding. symbio_ane/build_text_encoder.py builds it into <home>/cache/text-encoder from a separate coremltools venv. Its output matches the torch model at cosine 0.99997.

Decision order, with think/no-think accuracy on 31 held-out messages:

source	runs on	score	latency
MiniLM vote	ANE	30/31	<1 ms
Apple Intelligence's on-device model	ANE	27/31	~0.7 s
CPU embedding vote	CPU	30/31	~8 ms
regex	—	19/31	—

In live turns on the 14B, the decision came from minilm-ane in 2 ms.