fix(serve): coarse decode caps, 2048 MoE prefill chunk and fewer per-step host reads on pre-M5 GPUs - #233

fix(serve): coarse decode caps, 2048 MoE prefill chunk and fewer per-step host reads on pre-M5 GPUs#233

asher merged 8 commits into

mainasher/gmlx:mainfrom

fix/release-052-regressionsasher/gmlx:fix/release-052-regressionsCopy head branch name to clipboard