A reinforcement learning framework that combines market data with real-time news sentiment to make intelligent trading decisions. Unlike traditional algorithmic trading systems that rely solely on price/volume data, this system incorporates news events, earnings surprises, social sentiment, and market-moving announcements as part of the state space.
- Multi-Source News Integration - Reuters, Bloomberg, SEC filings, Twitter, Reddit, StockTwits
- FinBERT Sentiment Analysis - Domain-specific NLP for financial text
- Event-Driven Trading - Special handling for earnings, FDA decisions, M&A announcements
- Advanced RL Algorithms - PPO and SAC with attention mechanisms
- Cross-Attention Fusion - Market and news features attend to each other
- Multiple Fusion Strategies - Concatenation, Gated, Attention, FiLM
- Comprehensive Backtesting - Realistic simulation with commission and slippage
- Paper Trading Ready - Live simulation with real-time news monitoring
| Metric | Value |
|---|---|
| Python Files | 93 |
| Lines of Code | 35,275 |
| Test Cases | 178 (100% passing) |
| Core Modules | 75 |
# Clone the repository
git clone https://github.com/yourusername/rl-trading-agent.git
cd rl-trading-agent
# Create virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install PyTorch (CPU version)
pip install torch
# Or for CUDA support
pip install torch --index-url https://download.pytorch.org/whl/cu121
# Install dependencies
pip install -r requirements.txt
# Download FinBERT model (optional, for sentiment analysis)
python scripts/download_models.py --model finbertpython scripts/train.py \
--config experiments/earnings_trading/config.yaml \
--timesteps 2000000 \
--output models/trained/news_ppo_v1python scripts/backtest.py \
--model models/trained/news_ppo_v1 \
--data data/processed/aapl_2023.parquet \
--news-data data/news_cache/AAPL/2023 \
--output results/backtest_resultspython scripts/paper_trade.py \
--model models/trained/news_ppo_v1 \
--broker alpaca \
--symbol AAPL \
--capital 100000 \
--news-sources reuters,twitterrl-trading-agent/
├── src/
│ ├── agents/ # RL agents (PPO, SAC)
│ │ ├── news_reactive_ppo.py
│ │ ├── news_reactive_sac.py
│ │ └── ensemble.py
│ ├── networks/ # Neural network modules
│ │ ├── market_encoder.py # MLP/LSTM encoders
│ │ ├── news_encoder.py # News feature encoder
│ │ ├── cross_attention.py # Cross-modal attention
│ │ └── fusion.py # Feature fusion layers
│ ├── environments/ # Trading environments
│ │ ├── news_trading_env.py # Gymnasium environment
│ │ ├── reward_functions.py
│ │ └── action_masking.py
│ ├── news/ # News processing
│ │ ├── fetchers/ # API integrations
│ │ │ ├── reuters.py
│ │ │ ├── sec_filings.py
│ │ │ ├── twitter.py
│ │ │ └── reddit.py
│ │ └── processing/ # NLP pipeline
│ │ ├── finbert.py
│ │ └── event_extractor.py
│ ├── data/ # Data handling
│ │ └── features/
│ │ └── technical.py # Technical indicators
│ ├── training/ # Training utilities
│ ├── evaluation/ # Backtesting & analysis
│ │ ├── backtester.py
│ │ └── visualizer.py
│ ├── execution/ # Paper/live trading
│ └── strategies/ # Trading strategies
├── scripts/ # CLI entry points
│ ├── train.py
│ ├── backtest.py
│ └── paper_trade.py
├── tests/ # Test suite (178 tests)
│ ├── unit/
│ ├── integration/
│ ├── news/
│ ├── backtest/
│ └── mock_news/
├── experiments/ # Configuration files
├── models/ # Saved models
└── data/ # Data storage
┌─────────────────────────────────────────────────────────────────┐
│ NEWS & SENTIMENT PIPELINE │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ Reuters │ │ SEC │ │ Social │ │ Earnings │ │
│ │ API │ │ Filings │ │ Media │ │ Calendar │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ └─────────────┴──────┬──────┴─────────────┘ │
│ ▼ │
│ ┌─────────────────────────────┐ │
│ │ FinBERT NLP Processing │ │
│ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ FEATURE FUSION LAYER │
│ Market Features + News Features → Combined State │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ PPO/SAC RL AGENT │
│ Cross-Attention Network → Policy → Trading Action [-1, +1] │
└─────────────────────────────────────────────────────────────────┘
Example configuration file (experiments/config.yaml):
experiment:
name: news_reactive_ppo_v1
seed: 42
news:
sources:
- name: reuters
enabled: true
- name: sec_filings
enabled: true
- name: twitter
enabled: true
min_followers: 10000
sentiment:
model: finbert
aggregation: time_weighted
decay_hours: 24
agent:
algorithm: ppo
network:
market_encoder:
type: lstm
hidden: 128
news_encoder:
type: transformer
hidden: 128
fusion:
type: cross_attention
ppo:
learning_rate: 0.0003
gamma: 0.99
clip_range: 0.2
n_steps: 2048
batch_size: 64
environment:
initial_balance: 100000
commission: 0.001
slippage: 0.0005| Event Type | Impact Duration | Typical Magnitude |
|---|---|---|
| Earnings Surprise | 3-5 days | High |
| FDA Approval/Rejection | 1-3 days | Very High |
| M&A Announcement | 1-2 days | Very High |
| Analyst Upgrade/Downgrade | 1-2 days | Medium |
| Guidance Update | 2-3 days | Medium-High |
| Social Media Viral | Hours | Variable |
# Run all tests
pytest tests/ -v
# Run specific test categories
pytest tests/unit -v # Unit tests
pytest tests/integration -v # Integration tests
pytest tests/news -v # News pipeline tests
pytest tests/backtest -v # Backtest validation
# Run with coverage
pytest tests/ --cov=src --cov-report=html============================= test session starts ==============================
collected 178 items
tests/backtest/test_backtest_validation.py ............... [ 8%]
tests/integration/test_training_pipeline.py .......... [14%]
tests/mock_news/test_mock_news.py ........... [20%]
tests/news/test_news_pipeline.py ................ [29%]
tests/unit/test_agents.py ................. [39%]
tests/unit/test_backtester.py .................... [50%]
tests/unit/test_finbert.py ............... [58%]
tests/unit/test_networks.py .................. [68%]
tests/unit/test_news_base.py .............. [76%]
tests/unit/test_technical_features.py .............. [84%]
tests/unit/test_trading_env.py .................. [100%]
============================= 178 passed in 3.29s ==============================
- Case Study - Business overview and project summary
- Technical Documentation - Deep dive into AI/ML architecture
- CLAUDE.md - Development guide and codebase instructions
# Core
torch>=2.2.0
numpy>=1.26.0
pandas>=2.2.0
# RL
gymnasium>=0.29.0
# NLP
transformers>=4.38.0
# Market Data
yfinance>=0.2.36
alpaca-trade-api>=3.0.0
# Testing
pytest>=8.0.0
| Metric | Description | Target |
|---|---|---|
| News Alpha | Return from news signals | > 5% annually |
| Event Capture Rate | Event moves captured | > 60% |
| Sentiment Accuracy | Correlation with returns | > 0.15 |
| Sharpe Improvement | vs price-only baseline | > 0.3 |
| Earnings Win Rate | Profitable earnings trades | > 55% |
To use all features, you'll need API keys for:
| Service | Purpose | Required |
|---|---|---|
| Alpaca | Market data & paper trading | Yes (for trading) |
| Twitter/X | Social sentiment | Optional |
| Reddit sentiment | Optional | |
| NewsAPI | News aggregation | Optional |
Set API keys as environment variables:
export ALPACA_API_KEY="your_key"
export ALPACA_SECRET_KEY="your_secret"
export TWITTER_API_KEY="your_key"
export REDDIT_CLIENT_ID="your_id"- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Run tests (
pytest tests/ -v) - Commit changes (
git commit -m 'Add amazing feature') - Push to branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
IMPORTANT: This is an educational project for learning reinforcement learning and natural language processing in the context of financial markets. It is NOT financial advice.
- Past performance does not guarantee future results
- Always conduct thorough testing before any live trading
- Paper trade extensively before considering real money
- News sentiment can be noisy and lead to false signals
- Social media data can be manipulated
- The authors assume no responsibility for financial losses
- OpenAI - PPO algorithm
- Hugging Face - Transformers library and FinBERT
- Gymnasium - RL environment interface
- PyTorch - Deep learning framework