In the fast-moving world of general insurance, pricing models must be frequently updated to reflect changing risks and data. However, these updates can cause models to change not just in their predictions but also in their underlying structure, potentially undermining trust and consistency. A newly published research paper introduces a novel way to measure how stable the structure of transparent AI models remains after updates—an advance that could help insurers maintain reliable and interpretable pricing tools over time.
Key Takeaways
- Insurance pricing models often require regular updates due to rapid shifts in data, which can lead to significant changes in model structure and predictions.
- The researchers propose “functional transparency,” a concept describing models whose predictions can be broken down into simple, interpretable components based on individual features and their pairwise interactions.
- They introduce a new metric called the Mean Squared Shape function distance to quantify structural stability in these transparent models.
- Testing on three public auto insurance datasets, the study finds that generalized linear models are highly stable, while explainable boosting machines (EBMs) offer a good balance between stability and predictive performance.
- Strategies like sequential modeling and fixing hyperparameters can enhance stability in both EBMs and more complex models like XGBoost, without hurting accuracy.
The research focuses on a challenge common in actuarial science and insurance: how to keep pricing models both accurate and interpretable as they adapt to new data. Traditional metrics often look at how predictions change after model updates but pay less attention to changes in the model’s internal structure—how it “thinks.” This structural stability is crucial because large shifts in model structure can make the model’s behavior harder to understand and trust, even if its predictions remain accurate.
To tackle this, the authors introduce the idea of functional transparency. In simple terms, a functionally transparent model is one where the overall prediction can be expressed as the sum of effects from individual features (like a driver’s age or car type) plus effects from pairs of features interacting (such as age combined with car type). This decomposition helps humans understand exactly how each factor contributes to the final price.
Building on this, the researchers propose a new way to measure how much a model’s structure changes after an update: the Mean Squared Shape function distance. This metric compares the “shape functions”—the individual and pairwise effects—before and after updating the model. A small distance means the model’s internal reasoning remains consistent, while a large distance signals structural shifts.
The study applies this metric to three publicly available auto insurance datasets, benchmarking different modeling approaches. Generalized linear models (GLMs), a traditional and well-understood method, showed strong structural stability but sometimes at the cost of predictive power. Explainable boosting machines (EBMs), a modern interpretable machine learning technique, struck a balance by offering reasonable stability alongside improved accuracy. More complex models like XGBoost tended to be less stable structurally but could be stabilized with certain modeling strategies.
To improve stability, the authors tested techniques such as sequential modeling—where models are updated in stages rather than all at once—and fixing hyperparameters, which are the settings that control model training. These approaches helped keep the model structure more consistent after updates without sacrificing performance.
This research offers a valuable tool for insurance companies and other industries relying on transparent AI models that must adapt frequently to new data. By quantifying structural stability, practitioners can better monitor and manage how their models evolve, maintaining both reliability and interpretability. Future work may explore applying these ideas to other types of transparent models and further refining the balance between stability and predictive accuracy in dynamic environments.
Based on research published on arXiv by Félix Côté, Marie-Pier Côté.