Researchers have revisited a decades-old mathematical approach to reasoning puzzles and found it dramatically outperforms newer, popular AI methods. The study, recently published on arXiv, shows that energy-based models—specifically a learned version of a classic technique from the 1980s—can solve complex puzzles like Sudoku and mazes thousands of times faster than current looped neural network models. This breakthrough could reshape how AI systems handle reasoning tasks, making them more efficient and interpretable.
Key Takeaways
- A modern adaptation of Hopfield & Tank’s energy-based model, called CRAM, solves 99.996% of difficult Sudoku puzzles and 100% of unique mazes tested.
- CRAM trains and solves puzzles in milliseconds (42 ms for Sudoku, 132 ms for mazes) on a single GPU—over 1,000 times faster than looped models.
- It uses at least 282 times fewer computational operations (FLOPs) during inference compared to the best looped models, making it far more efficient.
- Energy-based reasoning models offer advantages like built-in early stopping, adaptive compute based on problem difficulty, and interpretable weights that reveal puzzle constraints.
Looped models have recently gained attention for tackling reasoning problems. These models repeatedly apply neural network computations in a loop to gradually refine answers, somewhat like iterating guesses until a solution emerges. While effective on simple puzzles, they often require long training times, many parameters, and heavy computational resources. Additionally, their iterative nature can make training unstable and slow.
The new research revisits energy-based models, a classical approach where the solution to a problem is found by minimizing an energy function—a mathematical expression designed so that its lowest value corresponds to the correct solution. The model “descends” the energy landscape, adjusting variables to find the lowest point, much like a ball rolling down a hill to settle in a valley. This process naturally includes early stopping if the solution is found early, and the compute needed adapts to how hard the problem is.
The authors developed CRAM, a learned energy function inspired by the Hopfield & Tank model from the 1980s. Unlike looped models, CRAM does not require differentiating through multiple solver steps during training, simplifying and speeding up the process. Its weights—the internal parameters of the model—are interpretable, meaning they directly capture the constraints of the puzzles (such as Sudoku’s rules), offering transparency often lacking in modern AI methods.
Testing CRAM on challenging benchmarks, the team found it solved nearly all Sudoku-Extreme puzzles and all Maze-Unique puzzles with remarkable speed and efficiency. These results far exceeded those of looped models, which struggled with speed and required significantly more computational power. The researchers caution that popular benchmarks like Sudoku and Maze, often considered trivial, may not be sufficient to test the scalability of looped reasoning methods, given how well energy models perform.
This work highlights the potential of energy-based models as a promising direction for training AI systems to reason effectively from data. By combining fast training, efficient inference, and interpretability, such models could improve applications ranging from puzzle solving to more complex decision-making tasks. Future research may explore scaling these techniques to even more challenging reasoning problems and integrating them into broader AI systems.
Based on research published on arXiv by Benjamin Hoover, Hugh Blayney, James C. R. Whittington.