PrimoGPT is a modular trading system developed as part of doctoral research titled "An Automated Stock Market Trading System Based on Deep Reinforcement Learning". The system integrates advanced Natural Language Processing (NLP) and Deep Reinforcement Learning (DRL) to enhance decision-making in capital markets, aiming to minimize risk and potential losses through deep financial market analysis while addressing challenges like market volatility and data complexity.
The research introduces two key components: PrimoGPT, a specialized Transformer-based NLP model for extracting features from financial texts (news, reports, etc.), and PrimoRL, a deep reinforcement learning model for making adaptive trading decisions. The system has demonstrated superior performance over traditional strategies in experimental evaluations on leading technology stocks (AAPL, NFLX, MSFT, CRM, AMZN) during the testing period from August 1, 2024, to February 28, 2025, consistently yielding above-market returns and high Sharpe ratios.
This project was created primarily for research and educational purposes as part of doctoral studies. It is not designed as a ready-to-use application at this stage. The codebase is structured to demonstrate concepts and methodologies rather than provide a turnkey trading solution. Users should be aware that:
- The system requires technical knowledge to set up and run
- Components are organized for research exploration rather than end-user experience
Future iterations will focus on improving usability and creating more accessible interfaces as outlined in the development roadmap, building upon the research findings that demonstrated the system's potential to outperform established strategies.
The core innovation lies in its PrimoGPT module, which utilizes a custom RAG (Retrieval Augmented Generation) system for feature generation from financial texts. This approach enables:
- Contextual decision making: Extracting structured features from unstructured financial news and reports provides critical context.
- Flexible feature generation: Generated NLP features can be used for diverse downstream tasks.
- Early AI agent approach: Developed in early 2024, it represents an early application of LLMs to financial analysis.
The next iteration of the system will focus on:
- Sentence embedding integration: Moving towards more flexible representations of financial text.
- Autonomous feature generation: Enabling LLMs to create relevant features independently.
- Component separation: Developing PrimoGPT and PrimoRL as standalone libraries.
- Library development: Creating easy-to-use libraries for integration.
- User interface: Building a simple Streamlit-based UI for wider accessibility.
- Primo Investing platform: Utilizing these libraries as the core for a public platform.
- Enhancing Architecture: Incorporating NLP features directly into the PrimoRL reward function, expanding to other asset classes, and refining PrimoGPT's feature generation capabilities.
This evolution aims to make the system more adaptable and effective at identifying significant patterns in financial data.
The project consists of two main components:
- Custom implementation of RAG (Retrieval Augmented Generation).
- Feature generation pipeline processing financial texts to extract NLP features (e.g., sentiment, relevance, potential price impact).
- Fine-tuned LLM model based on Meta-Llama-3.1-8B-Instruct, trained using instruction-based datasets generated by GPT-4 and incorporating next-day stock price movements during training (not used during operational prediction).
- Key implementation details in notebooks:
notebooks/1. Get and process data/: Data collection (Finnhub, Yahoo Finance) and preprocessing pipeline (based on FinGPT Forecaster).notebooks/2. Train PrimoGPT model/: Model fine-tuning using Unsloth and QLoRA.notebooks/5. Generate NLP features with PrimoGPT/: NLP feature extraction pipeline using custom prompts via LangChain.
- Deep Reinforcement Learning trading framework built using OpenAI Gymnasium and FinRL.
- Implements Stable Baselines 3 algorithms (A2C, SAC, PPO).
- Utilizes technical indicators (MACD, Bollinger Bands, RSI, DX with 30-day period; SMA with 30 and 60-day periods) calculated from Yahoo Finance data.
- Features three custom trading environments extending FinRL:
env_primo_default: Enhanced stock trading environment with custom logging.env_primo_default_nlp: Extended environment with NLP capabilities.env_primorl: New environment incorporating NLP features into the state space and using a custom reward function balancing portfolio return and Sharpe ratio.
- Core implementation in
src/primorl/. - Training and experimentation in
notebooks/6. PrimoRL trading with NLP features/.
The project is organized into notebooks for experimentation and source code for the core implementation:
notebooks/
├── 1. Get and process data/ # Data collection and preprocessing scripts
├── 2. Train PrimoGPT model/ # PrimoGPT model fine-tuning
├── 3. Trading benchmark strategy/ # Implementation of baseline trading strategies
├── 4. FinRL benchmark strategy/ # Implementation of FinRL trading strategies
├── 5. Generate NLP features with PrimoGPT/ # NLP feature extraction pipeline
└── 6. PrimoRL trading with NLP features/ # Combined NLP-RL trading experiments
src/
├── primogpt/ # Core PrimoGPT implementation
└── primorl/ # Core PrimoRL trading environments
The system has been validated through extensive testing on five technology stocks (AAPL, NFLX, MSFT, CRM, AMZN) with:
- Training period: April 1, 2022, to July 31, 2024
- Testing period: August 1, 2024, to February 28, 2025
The PrimoGPT model was trained on a separate dataset (AMD, GOOGL, IBM, META, TSLA) to ensure generalizability.
The effectiveness of PrimoRL, particularly when incorporating NLP features from PrimoGPT, was evaluated against traditional strategies (Buy & Hold, Momentum, Price-MA, MACD), market benchmarks (DJI), optimized strategies (Mean-Variance), and the baseline FinRL framework across the testing period.
| Stock | Model | Cumulative Return | Sharpe Ratio | Annualized Volatility | Maximum Drawdown |
|---|---|---|---|---|---|
| AAPL | PrimoRL | 12.20% | 1.15 | 19.08% | -12.14% |
| MOM | 21.20% | 2.16 | 16.22% | -9.37% | |
| NFLX | PrimoRL | 62.82% | 2.83 | 32.07% | -7.74% |
| B&H | 58.12% | 2.62 | 32.82% | -10.90% | |
| MSFT | PrimoRL | 1.45% | 0.22 | 21.24% | -9.89% |
| FinRL | 0.24% | 0.12 | 21.03% | -9.48% | |
| CRM | PrimoRL | 31.95% | 1.64 | 33.05% | -14.55% |
| P-MA | 22.36% | 1.77 | 21.27% | -11.53% | |
| AMZN | PrimoRL | 21.48% | 1.42 | 26.60% | -11.90% |
| FinRL | 20.30% | 1.35 | 26.64% | -11.90% |
PrimoRL generally outperformed benchmarks, showing significant gains especially on NFLX, CRM, and AMZN, though MOM strategy was superior for AAPL.
| Model | Cumulative Return | Sharpe Ratio | Annualized Volatility | Maximum Drawdown |
|---|---|---|---|---|
| Mean Variance | 22.24% | 1.64 | 23.19% | -9.22% |
| DJI (B&H) | 6.35% | 0.94 | 12.24% | -6.91% |
| FinRL (A2C) | -4.83% | -0.25 | 23.55% | -11.80% |
| FinRL (SAC) | 24.50% | 1.28 | 34.53% | -16.50% |
| FinRL (PPO) | 13.54% | 0.87 | 31.38% | -18.52% |
| PrimoRL (A2C) | 20.28% | 1.10 | 35.05% | -16.48% |
| PrimoRL (SAC) | 27.39% | 1.79 | 25.55% | -11.44% |
| PrimoRL (PPO) | -8.03% | -0.46 | 24.25% | -11.73% |
PrimoRL (SAC) demonstrated the best portfolio performance, achieving the highest cumulative return and Sharpe ratio with controlled volatility and drawdown, significantly outperforming FinRL variants and traditional benchmarks. This highlights the benefit of integrating NLP features, especially with the SAC agent.
The following graphs further illustrate the performance comparison:
Cumulative returns of FinRL variants (A2C, SAC, PPO) compared to Mean-Variance and DJI benchmarks, highlighting the superior adaptability of RL-based strategies.
Cumulative returns of PrimoRL variants (A2C, SAC, PPO) compared to DJI and Mean-Variance benchmarks, highlighting PrimoRL (SAC) as the optimal strategy with superior returns and controlled risk exposure.
This research builds upon and extends the excellent work of the AI4Finance Foundation. Their open-source projects served as the foundation and primary inspiration for this research. Special thanks for their groundbreaking work on:
- FinRL - The base framework for our reinforcement learning implementation
- FinGPT - Inspiration for financial NLP components
The success of this research would not have been possible without these foundational contributions to the field of AI in finance.
This project is licensed under the MIT License. This repository contains academic research code intended for educational purposes only. Nothing herein constitutes financial advice or trading recommendations. The trading strategies and analyses are experimental in nature and have not been validated for real-world trading. Users should be aware that trading involves substantial risk of loss and should always consult financial professionals before making investment decisions.