When artificial intelligence (AI) systems make decisions, they often rely on gathering a lot of information, or “features,” about each case they encounter. However, collecting every possible piece of data can be expensive, slow, or even invasive, especially in fields like healthcare where tests can be costly or uncomfortable. A newly published research paper introduces a smarter way for AI to decide which pieces of information to collect, aiming to save resources while maintaining strong prediction accuracy.
Key Takeaways
- The researchers developed a method called OracleImitator that learns from an “oracle” teacher to decide which data features to acquire for each case.
- OracleImitator outperforms or matches existing methods in selecting features efficiently, achieving 90% of the potential accuracy improvement at less than 25% of the usual data acquisition cost.
- The approach explicitly ranks features by their impact on prediction errors, unlike previous methods that rely on indirect optimization goals.
- This explicit ranking allows for better understanding and measurement of how well the AI’s feature selection policy works in real-world scenarios where not all information is visible upfront.
Traditional AI models often assume all relevant data about a case is readily available, but in practice, gathering every possible detail can be impractical. “Active Feature Acquisition” (AFA) is an area of research that seeks to tackle this problem by teaching AI systems to selectively acquire only the most useful features based on a given budget—be it money, time, or privacy concerns.
In this new study, the authors propose a fresh take on AFA. They introduce an “oracle” teacher that knows the true answer and all unobserved data during the training phase. This oracle can precisely calculate which features would most improve the AI’s predictions by directly measuring the effect of each feature on the model’s error. This explicit ranking acts like a gold standard or guide for the AI system to learn which features to prioritize.
The challenge is that at deployment time, the AI does not have access to this oracle or the hidden data—it only sees features it has already acquired. To overcome this, the researchers developed OracleImitator, which trains a policy by “distilling” the oracle’s rankings into a form the AI can use in practice. This is done through “soft-target imitation,” a technique where the AI learns to mimic the oracle’s guidance as closely as possible based on the information available.
To test their approach, the team evaluated OracleImitator on tasks including multimodal medical diagnosis, where data comes from different types of tests and measurements, and on a standard benchmark called AFABench. The results showed that OracleImitator either matched or exceeded existing methods across a range of budget levels. Impressively, it achieved most of the accuracy gains possible with far fewer features, effectively reducing costs and effort.
Because the oracle teacher is an explicit algorithm rather than an indirect optimization target, the authors can directly assess how well the learned policy approximates the ideal feature ranking. This clarity offers valuable insights into the trade-offs involved and the limits of what can be achieved given the information constraints at deployment.
This research marks a promising step toward more efficient AI systems that can make smarter decisions about which data to collect. In settings like healthcare, finance, or any domain where data acquisition is costly or sensitive, such methods could lead to faster, cheaper, and less intrusive AI-driven decisions. Future work may explore extending this approach to even more complex scenarios and integrating it into real-world applications where budgets and privacy are critical concerns.
Based on research published on arXiv by Yang Meng, Zixin Ding, Robert L. Grossman et al..
