We develop algorithms and architectures that put probabilistic computing to work in generative AI. We start from the computation a model needs and investigate which parts can benefit from fast sampling. This guides the design of p-computers and motivates algorithms that use their capabilities.
Generative AI with probabilistic computing
Diffusion models combine neural-network evaluation with stochastic sampling. We investigate how the sampling process itself can carry information about the system being modeled. In our work on correlated diffusion, known interactions guide the stochastic dynamics, while a neural network estimates the underlying clean state. Our initial demonstrations generate configurations of Ising models and spin glasses. This work explores how neural networks and p-computers could share the computational work of generative modeling.
Figure: Singh et al., From Independent to Correlated Diffusion, Fig. 1.
SANTA and SANTA++ for efficient attention
We also develop probabilistic methods that reduce memory traffic during large language model inference. SANTA samples from attention probabilities and combines only the selected value vectors. SANTA++ uses representative keys to select groups of cached tokens without first reading every key. It evaluates attention within the sampled groups and reweights their contributions using their inclusion probabilities. Both methods let us adjust the sampling budget to balance memory access against approximation accuracy. We implement these methods on GPUs and measure their effects on model accuracy and inference speed. These methods also guide our investigation of hardware designed for sparse, additive computation.
Scalable architectures
As p-computers grow, communication between devices can become slower than local updates. We build programmable p-computers that place interaction weights near the circuits that use them. Our recent system connects one million p-bits across 18 FPGAs. We study how communication between devices affects computation, and develop methods that divide large problems into smaller, interacting clusters.
Optimization and scientific computing
Generative models can also help probabilistic computers explore difficult optimization problems. We study learned proposals that move between distant configurations, alongside parallel tempering and variational optimization methods. For quantum many-body physics, we use p-computers to accelerate sampling in neural quantum states. These applications also help us understand where probabilistic methods are useful and where limitations such as the sign problem remain.
Figure: Delacour et al., Probabilistic algorithms for Ising machines from optimization to generative AI, Fig. 4.
Selected Recent Papers
-
Probabilistic algorithms for Ising machines from optimization to generative AI. Nature Reviews Electrical Engineering (2026).
-
Stochastic Sparse Attention for Memory-Bound Inference (SANTA). International Conference on Machine Learning (ICML), 2026.
-
SANTA++: Sampling Attention through Representative Keys. Preprint (2026).
-
Programmable Probabilistic Computer with 1,000,000 p-bits. Preprint.
-
IsingFormer: Augmenting Parallel Tempering With Learned Proposals. Preprint.
-
Probabilistic Computers for Neural Quantum States. Preprint.
Foundational Publications
- N. A. Aadit, A. Grimaldi, M. Carpentieri, L. Theogarajan, J. M. Martinis, G. Finocchio, K. Y. Camsari, Massively Parallel Probabilistic Computing with Sparse Ising Machines, Nature Electronics, (2022)
-
B. Sutton, R. Faria, L. A. Ghantasala, R. Jaiswal, K. Y. Camsari, S. Datta, Autonomous Probabilistic Coprocessing with Petaflips per Second, IEEE Access, (2020)
-
S. Chowdhury, K. Y. Camsari, S. Datta, Emulating Quantum Interference with Generalized Ising Machines, arXiv: 2007.07379, (2020)
-
K. Y. Camsari, S. Chowdhury, S. Datta, Scalable Emulation of Sign-Problem–Free Hamiltonians with Room Temperature p-bits, Physical Review Applied 12, 034061 (2019)
-
K. Y. Camsari, B. M. Sutton, S. Datta, p-Bits for Probabilistic Spin Logic, Applied Physics Reviews, 6, 011305 (2019)
-
A. Z. Pervaiz, B. M. Sutton, L. A. Ghantasala, K. Y. Camsari, Weighted p-bits for FPGA implementation of probabilistic circuits, IEEE Transactions on Neural Networks and Learning Systems, Vol. 30, 6, 1920-1926 (2018)
-
K. Y. Camsari, R. Faria, B. Sutton, S. Datta, Stochastic p-bits for invertible logic, Physical Review X, 7, 031014 (2017)