Computing Sensitivity of Certain Neural Networks Is Harder Than Expected, New Study Finds

Understanding how sensitive a neural network is to small changes in its input is crucial for ensuring its reliability and robustness, especially in safety-critical applications like autonomous driving or medical diagnosis. A common way to measure this sensitivity is through something called the Lipschitz constant, which essentially quantifies the maximum amount the network’s output can change in response to small input tweaks. However, calculating this constant exactly is known to be challenging for many kinds of neural networks. A newly published research paper dives into this problem for a specific class of networks called two-layer input-convex neural networks (ICNNs), revealing surprising complexity results that clarify why computing these sensitivity measures is so difficult.

Key Takeaways

• Calculating the exact Lipschitz constant for two-layer ICNNs with respect to the L_p-norm (a way to measure distances) is computationally hard when p is any rational number greater than 1 and less than infinity.
• This problem is equivalent to maximizing the L_p-norm over a mathematical shape called a zonotope, which generalizes certain geometric objects formed by combining line segments.
• While special cases like the L₁– and L_∞-norms have known efficient algorithms, the new results show that for other L_p-norms, no significantly faster algorithms are likely to exist under widely believed computational assumptions.
• The findings settle an open problem from a major machine learning theory conference and provide a clearer mathematical understanding of why these computations are so challenging.

The researchers focused on input-convex neural networks, which are a restricted but practically relevant type of neural network where the output function is convex—meaning it curves upwards and has no dips. This convexity is enforced by requiring certain weights in the network to be nonnegative. Two-layer ICNNs are relatively simple compared to deep networks but are still powerful enough to be useful in applications like energy-based models and structured prediction.

To study the Lipschitz constants for these networks, the authors translated the problem into a geometric one: maximizing the L_p-norm over zonotopes. Zonotopes are shapes formed by adding together line segments in multiple dimensions. Imagine stacking and combining sticks in different directions to create a higher-dimensional shape. The maximum L_p-norm over this shape corresponds to the Lipschitz constant of the neural network.

While earlier work had shown that for the L₁-norm (which sums absolute values) and L_∞-norm (which takes the maximum absolute value), there are efficient algorithms to solve this maximization, the complexity for other L_p-norms (like the common L₂-norm used in Euclidean distance) was unknown.

The new paper proves that for any fixed rational number p strictly between 1 and infinity, the problem is what computer scientists call W[1]-hard when parameterized by the dimension of the space—basically meaning it is very unlikely that an algorithm exists that can solve the problem quickly as the dimension grows. The authors also argue that the best brute-force methods currently known are essentially optimal under the Exponential Time Hypothesis, a widely accepted assumption in computational complexity theory.

To arrive at these conclusions, the researchers first established the hardness result for the L₂-norm case and then extended it to other L_p-norms using a mathematical technique called Taylor approximation, which approximates functions using polynomials. Their work not only settles a question posed at the Conference on Learning Theory (COLT) in 2025 but also provides a detailed, clear explanation of the underlying mathematics and conceptual ideas behind the proofs. Notably, they also describe how they incorporated large language models (LLMs) during their research process, showcasing a modern approach to tackling complex theoretical problems.

These findings have important implications for researchers and practitioners working on neural network robustness and certification. Knowing the inherent computational difficulty of exactly computing Lipschitz constants for these networks guides future work towards approximation methods or alternative architectures where sensitivity can be measured more efficiently. As neural networks continue to be deployed in critical systems, understanding these fundamental limits helps set realistic expectations and directs efforts towards practical solutions. The next steps may involve exploring efficient heuristics or focusing on subclasses of networks where the problem becomes more tractable.

Based on research published on arXiv by Aritra Das, Vincent Froese, Moritz Grillo et al..