Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer

@misc{mahendrakar2026neuromorphic,
  author       = {Mahendrakar, Pranay},
  title        = {{Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer}},
  year         = {2026},
  month        = apr,
  date         = {2026-04-28},
  publisher    = {Life of Research},
  doi          = {10.5281/zenodo.19854579},
  url          = {https://lifeofresearch.com/articles/7},
  abstract     = {Spiking neural networks (SNNs) on neuromorphic hardware are routinely proposed as the path to energy-efficient on-device inference for language models. The framing usually treats this as a single technical question. It is not. There are three distinct technical layers — SNN algorithms for language tasks, neuromorphic hardware platforms, and the integration of the two into actual deployed systems — and each layer has a different state of progress, a different set of obstacles, and a different research community. This paper makes three claims. First, conflating the layers obscures where the genuine gaps lie: SNN-LLM algorithm research has accelerated substantially with SpikeGPT, SpikeLLM, Sorbet, and recent spike-driven LLM constructions, and neuromorphic hardware has matured with Loihi 2, IBM NorthPole, SpiNNaker 2, and BrainChip Akida — but actual deployed SNN-LLM systems running on neuromorphic chips at the edge remain almost nonexistent. The integration layer is the bottleneck, not the algorithms. Second, the energy-efficiency case for neuromorphic-LLM deployment must be made against the right baseline — INT4 or sub-4-bit quantized models running on modern mobile NPUs — not against unquantized FP16 GPU baselines. The honest comparison narrows the advantage substantially and changes which workloads are worth pursuing. Third, the most plausible near-term wins are not as drop-in replacements for mobile-NPU LLM inference but in specific niches: always-on low-rate token processing, sensor-fused language tasks where input is already event-based, and ultra-low-power deployment regimes where mobile NPUs do not operate. We propose a research agenda focused on the integration gap and on rigorous baseline comparisons.},
  language     = {en},
  copyright    = {Creative Commons Attribution 4.0}
}