Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ path = "src/lib.rs"
name = "sql-cli"
path = "src/main.rs"

[[bin]]
name = "sql-cli-chart"
path = "src/chart_main.rs"



# Windows-specific dependencies
Expand All @@ -58,6 +62,7 @@ csv = "1.3"
ratatui = "0.29"
tui-input = { version = "0.11", features = ["crossterm"] }
anyhow = "1.0"
clap = "4.0"
regex = "1.0"
fuzzy-matcher = "0.3"
chrono = { version = "0.4", features = ["serde"] }
Expand Down
67 changes: 67 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,73 @@ FROM products
WHERE name.StartsWith('A')
```

## 📊 Terminal Charts (NEW!)

SQL CLI now includes a powerful **standalone charting tool** (`sql-cli-chart`) that creates terminal-based visualizations of your SQL query results. Perfect for time series analysis, trend visualization, and data exploration.

### Chart Tool Usage

```bash
# Basic time series chart
sql-cli-chart data.csv -q "SELECT time, value FROM data" -x time -y value -t "My Chart"

# Filter data with SQL WHERE clause
sql-cli-chart trades.csv \
-q "SELECT timestamp, price FROM trades WHERE symbol = 'AAPL'" \
-x timestamp -y price -t "AAPL Price Chart"
```

### Real-World Example: VWAP Trading Analysis

Visualize algorithmic trading data with SQL filtering to focus on specific patterns:

```bash
# Chart fill volume progression for CLIENT orders only
sql-cli-chart data/production_vwap_final.csv \
-q "SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \
-x snapshot_time -y filled_quantity \
-t "CLIENT Order Fill Progression"

# Compare with ALL orders (shows chaotic "Christmas tree" pattern)
sql-cli-chart data/production_vwap_final.csv \
-q "SELECT snapshot_time, filled_quantity FROM production_vwap_final" \
-x snapshot_time -y filled_quantity \
-t "All Orders - Mixed Pattern"
```

**The Power of SQL Filtering**: The first query filters to show only CLIENT orders (991 rows), displaying a clean upward progression. The second shows all 3320 rows including ALGO and SLICE orders, creating a noisy pattern. This demonstrates how SQL queries let you focus on exactly the data patterns you want to visualize.

### Interactive Chart Controls

Once the chart opens, use these vim-like controls:
- **hjkl** - Pan left/down/up/right
- **+/-** - Zoom in/out
- **r** - Reset view to auto-fit
- **q/Esc** - Quit

### Example Scripts

Ready-to-use chart examples are in the `scripts/` directory:

```bash
# VWAP average price over time
./scripts/chart-vwap-price.sh

# Fill volume progression
./scripts/chart-vwap-volume.sh

# Compare different order types
./scripts/chart-vwap-algo-comparison.sh
```

### Chart Features

- **SQL Query Integration**: Use full SQL power to filter and transform data before charting
- **Smart Auto-Scaling**: Automatically adapts Y-axis range for optimal visibility
- **Time Series Support**: Automatic timestamp parsing and time-based X-axis
- **Interactive Navigation**: Pan and zoom to explore your data
- **Terminal Native**: Pure terminal graphics, no GUI dependencies

## 🔧 Development

### Running Tests
Expand Down
137 changes: 137 additions & 0 deletions docs/CHARTING_QUICKSTART.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# SQL CLI Charting - Quick Start

## Overview

SQL CLI now includes a **standalone charting tool** (`sql-cli-chart`) for creating terminal-based visualizations of CSV/JSON data. This is perfect for quick data exploration and analysis without needing external tools.

## Features

- **Terminal-native**: Pure ratatui-based charts, no GUI dependencies
- **Time series support**: Automatic timestamp parsing and scaling
- **Interactive controls**: Vim-like navigation (hjkl), pan, zoom
- **Smart auto-scaling**: Adapts Y-axis ranges for optimal data visibility
- **SQL engine integration**: Leverages the full SQL query capabilities

## Installation

The chart tool is built as a separate binary alongside the main sql-cli:

```bash
cargo build --release
# Creates both binaries:
# - ./target/release/sql-cli (main TUI)
# - ./target/release/sql-cli-chart (charting tool)
```

## Basic Usage

```bash
sql-cli-chart <file.csv> -q "SQL_QUERY" -x X_COLUMN -y Y_COLUMN -t "Chart Title"
```

### Parameters

- **file.csv**: Input CSV or JSON file
- **-q, --query**: SQL query to filter/transform data (required)
- **-x, --x-axis**: Column name for X-axis (required)
- **-y, --y-axis**: Column name for Y-axis (required)
- **-t, --title**: Chart title (optional, default: "SQL CLI Chart")
- **-c, --chart-type**: Chart type - line, scatter, bar (optional, default: line)

## Examples

### Time Series: VWAP Price Analysis

```bash
# Basic price over time
./target/release/sql-cli-chart data/production_vwap_final.csv \\
-q "SELECT snapshot_time, average_price FROM production_vwap_final WHERE filled_quantity > 0" \\
-x snapshot_time \\
-y average_price \\
-t "VWAP Average Price Over Time"

# Volume analysis
./target/release/sql-cli-chart data/production_vwap_final.csv \\
-q "SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \\
-x snapshot_time \\
-y filled_quantity \\
-t "Client Order Fill Volume"
```

### Scatter Plot: Price vs Volume Correlation

```bash
./target/release/sql-cli-chart data/production_vwap_final.csv \\
-q "SELECT average_price, filled_quantity FROM production_vwap_final WHERE filled_quantity > 0" \\
-x average_price \\
-y filled_quantity \\
-c scatter \\
-t "Price vs Volume Correlation"
```

## Interactive Controls

Once the chart opens, use these vim-like controls:

- **hjkl**: Pan left/down/up/right
- **+/-**: Zoom in/out
- **r**: Reset view to auto-fit data
- **q/Esc**: Quit

## Smart Features

### Auto-Scaling
The tool automatically:
- Detects timestamp columns and converts to epoch time for plotting
- Applies smart Y-axis scaling with 10% padding
- Handles different numeric ranges (prices vs quantities)

### Time Series Support
Timestamps in RFC3339 format (like `2025-08-12T09:00:18.030000`) are:
- Automatically detected and parsed
- Converted to epoch seconds for plotting
- Displayed as HH:MM:SS on the X-axis

## Architecture

The charting system is designed as a **standalone subsystem**:

- **Non-invasive**: Separate binary, doesn't interfere with main TUI
- **Modular**: Clean separation in `src/chart/` module
- **Extensible**: Easy to add new chart types and features
- **Reusable**: Can be integrated into main TUI later

## Current Limitations

1. **No SQL query execution yet**: Currently works with raw data (query parameter ignored for now)
2. **Terminal compatibility**: May have issues in non-TTY environments
3. **Limited chart types**: Only line charts fully implemented

## Roadmap

- [ ] Integrate SQL query execution
- [ ] Add candlestick charts for OHLC data
- [ ] Export chart data to CSV
- [ ] Multiple data series on same chart
- [ ] Integration with main TUI (press 'C' for chart mode)

## Technical Details

### Data Flow
```
CSV/JSON → DataTable → DataView → SQL Query → Chart Data → Ratatui Rendering
```

### Key Components
- `ChartEngine`: Data processing and query execution
- `LineRenderer`: Time series visualization
- `ChartTui`: Interactive terminal interface
- `ChartViewport`: Pan/zoom state management

## Troubleshooting

**"No such device or address" error**: This occurs in environments without proper TTY support. The chart tool requires a real terminal for interactive display.

**Empty chart**: Check that your SQL query returns data and column names match exactly.

**Poor scaling**: Use the 'r' key to reset auto-scaling, or manually pan/zoom to find your data.
60 changes: 60 additions & 0 deletions scripts/chart-vwap-algo-comparison.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
#!/bin/bash

# Compare ALGO vs CLIENT orders
# This script shows different queries to compare order types

# Navigate to project root
cd "$(dirname "$0")/.." || exit 1

# Check if binary exists
if [ ! -f "./target/release/sql-cli-chart" ]; then
echo "Error: sql-cli-chart binary not found."
echo "Please run 'cargo build --release' first."
exit 1
fi

echo "=== VWAP Order Type Comparison ==="
echo ""
echo "Choose which view to display:"
echo "1) CLIENT orders only (clean progression)"
echo "2) ALGO_PARENT orders (algorithm's parent orders)"
echo "3) ALGO_SLICE orders (algorithm's child slices)"
echo "4) All orders (Christmas tree pattern)"
echo ""
read -p "Enter choice (1-4): " choice

case $choice in
1)
QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'"
TITLE="CLIENT Orders - Fill Progression"
;;
2)
QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type = 'ALGO_PARENT'"
TITLE="ALGO Parent Orders - Fill Progression"
;;
3)
QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type = 'ALGO_SLICE'"
TITLE="ALGO Slice Orders - Fill Progression"
;;
4)
QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE filled_quantity > 0"
TITLE="All Orders - Christmas Tree Pattern"
;;
*)
echo "Invalid choice. Exiting."
exit 1
;;
esac

echo ""
echo "Executing: $QUERY"
echo ""

./target/release/sql-cli-chart data/production_vwap_final.csv \
-q "$QUERY" \
-x snapshot_time \
-y filled_quantity \
-t "$TITLE"

echo ""
echo "Chart closed."
27 changes: 27 additions & 0 deletions scripts/chart-vwap-price.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash

# Chart VWAP Average Price Over Time (CLIENT orders only)
# This script visualizes how the VWAP average price evolves for client orders

# Navigate to project root
cd "$(dirname "$0")/.." || exit 1

# Check if binary exists
if [ ! -f "./target/release/sql-cli-chart" ]; then
echo "Error: sql-cli-chart binary not found."
echo "Please run 'cargo build --release' first."
exit 1
fi

echo "=== VWAP Average Price Chart ==="
echo "Visualizing CLIENT order average price progression over time"
echo ""

./target/release/sql-cli-chart data/production_vwap_final.csv \
-q "SELECT snapshot_time, average_price, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \
-x snapshot_time \
-y average_price \
-t "CLIENT Order VWAP Price Over Time"

echo ""
echo "Chart closed."
28 changes: 28 additions & 0 deletions scripts/chart-vwap-volume.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#!/bin/bash

# Chart VWAP Fill Volume Progression (CLIENT orders only)
# This script visualizes how the filled quantity accumulates for client orders

# Navigate to project root
cd "$(dirname "$0")/.." || exit 1

# Check if binary exists
if [ ! -f "./target/release/sql-cli-chart" ]; then
echo "Error: sql-cli-chart binary not found."
echo "Please run 'cargo build --release' first."
exit 1
fi

echo "=== VWAP Fill Volume Chart ==="
echo "Visualizing CLIENT order fill quantity progression"
echo "Expected: Smooth upward trend starting from 0"
echo ""

./target/release/sql-cli-chart data/production_vwap_final.csv \
-q "SELECT snapshot_time, average_price, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \
-x snapshot_time \
-y filled_quantity \
-t "CLIENT Order Fill Volume Progression"

echo ""
echo "Chart closed."
Loading
Loading