Managing large investment portfolios is a challenging task, especially when strict rules govern how assets must be allocated under different market conditions. While advanced AI techniques like deep reinforcement learning (RL) have shown promise in optimizing portfolio returns, they often struggle to consistently obey these complex constraints, limiting their practical use. A newly published research paper introduces a novel neurosymbolic RL approach that enables AI agents to internalize and follow detailed investment mandates with far fewer violations, potentially paving the way for safer AI-driven portfolio management.
Key Takeaways
- The study develops a continuous-action neurosymbolic reinforcement learning method that learns to comply with investment policy rules without external corrections during deployment.
- Compared to unconstrained RL agents, which violated rules up to 98% of the time, the new approach reduced violations to under 5%.
- The method estimates the likelihood that each action satisfies the mandate using advanced sampling techniques and incorporates this directly into training.
- This approach improves on previous neurosymbolic RL methods by handling continuous portfolio allocations instead of just discrete choices.
Investment portfolios for institutions like pension funds or mutual funds usually operate under detailed investment policy statements. These documents specify how assets should be allocated—for example, limits on certain asset classes or risk levels—and often change depending on the market environment. Traditional AI approaches to portfolio management, such as deep reinforcement learning, excel at learning strategies that maximize financial returns by trial and error. However, they typically do not guarantee adherence to these complex, rule-based constraints, which is a critical requirement in real-world financial applications.
The researchers behind this new study addressed this challenge by building on a neurosymbolic approach called Probabilistic Logic Policy Gradient (PLPG). Neurosymbolic methods combine neural networks (which excel at learning from data) with symbolic reasoning (which encodes explicit rules or logic). Previous PLPG techniques could enforce rules but were limited to scenarios where the AI’s possible actions were discrete and finite—such as choosing between a small set of predefined options. Portfolio allocation, however, involves continuous decisions, like deciding exactly what percentage of capital to assign to each asset.
To tackle this, the team developed a continuous-action version inspired by PLPG. Their method uses a technique called reparameterized Monte Carlo sampling to estimate the probability that the action chosen by the AI will comply with the investment rules. This compliance probability is then incorporated directly into the AI’s learning objective, effectively teaching the AI to prefer actions that follow the mandate. Importantly, this learning happens internally, so at deployment time, no external rule-checking or correction is needed—the AI’s policy has already internalized the constraints.
Testing their approach on a portfolio allocation task governed by six regime-conditional rules, the researchers found dramatic improvements. Unconstrained RL agents violated the rules 96-98% of the time, while a baseline method that applied penalties for violations reduced this to about 33%. Their neurosymbolic continuous-action policy further reduced violations to just 4.8%, demonstrating strong internalization of the mandate. Moreover, the policy’s actions were audited directly without any post-hoc adjustments, confirming that compliance was learned rather than enforced externally.
This research suggests a promising direction for bringing advanced AI into regulated financial environments, where rule compliance is as important as performance. By combining symbolic logic with continuous-action reinforcement learning, AI agents can better respect complex investment mandates, potentially increasing trust and safety in automated portfolio management. Future work may explore scaling this approach to even more intricate rules, diverse asset classes, or real-time market data, moving closer to practical deployment in institutional finance.
Based on research published on arXiv by Zeynep Yıldız, Satchit Chatterji, Ercan Atam et al..