Understanding why artificial intelligence (AI) models make certain decisions has long been a challenge, especially with complex language models that generate human-like text. A newly published research paper introduces a novel approach that combines principles from quantum physics with AI to provide exact, mathematically precise explanations for how language models arrive at their answers. This advancement could help make AI systems more transparent and trustworthy by revealing the exact contribution of each word in a model’s decision.
Key Takeaways
- The researchers developed a new transformer language model that uses “Born measurements” from quantum theory to calculate attention, a key component in how models focus on different words.
- This model achieves comparable accuracy (about 77%) on a natural language inference task, matching traditional transformer models of similar size.
- The approach allows exact computation of how much each word contributes to the model’s final decision, with extraordinarily low error margins (less than one part in 1011).
- They demonstrated that the same model can be implemented on quantum hardware, running tests on IBM’s quantum computer which replicated the model’s decision patterns on simple examples.
Traditional language models use an “attention” mechanism to weigh the importance of words when generating answers or predictions. However, these attention scores are often heuristic and do not provide exact explanations for the model’s decisions. The new approach replaces standard attention calculations with a “Born measurement,” a concept borrowed from quantum physics that involves probabilities derived from the square of wave function amplitudes.
In this quantum-inspired transformer, each step of processing language data corresponds to a Born measurement, where attention is calculated using the fourth power of the overlap between query and key vectors (mathematical representations of words). This method allows the model to represent the probability of each output class (such as an answer to a question) as an exact polynomial function of the input word amplitudes. Because of this precise mathematical form, the researchers can compute the contribution of each word to the final decision without approximation errors common in other explanation methods.
The team tested their model on the e-SNLI dataset, which involves natural language inference tasks — essentially, determining whether a statement logically follows from another. Their model, with around 98,000 parameters excluding word embeddings, reached an accuracy of 76.98%, on par with a traditional transformer using a softmax function for attention. More importantly, when they removed the word that their explanation method identified as most influential, the model’s answer changed in 45% of cases where any single word deletion could alter the outcome. This is a stronger alignment between explanation and behavior than seen in standard models.
Another exciting aspect of this research is its connection to quantum computing. The authors show how their model’s computations can be mapped exactly onto a quantum circuit using 23 qubits and 852 quantum gates (called CNOTs). They ran initial experiments on IBM’s quantum computer, successfully replicating the model’s decisions on small test cases by measuring quantum overlaps. While the full explanation evidence was too delicate to survive current quantum hardware noise, the researchers outline what improvements in hardware and calibration would be needed to measure it directly in the future.
This research opens up intriguing possibilities for AI transparency and quantum computing. By providing exact, mathematically grounded explanations for language model outputs, it could help developers and users better understand AI decisions, potentially increasing trust and safety. Additionally, the demonstration that such models can run on quantum devices hints at future hybrid AI-quantum systems that leverage the strengths of both fields. Further work will be needed to scale these methods to larger models and more complex tasks, and to realize robust quantum hardware capable of supporting precise AI explainability.
Based on research published on arXiv by Teerawat Tongloy, Jiraporn Pongsopa, Rangsan Jomtarak.