diff --git a/Cargo.toml b/Cargo.toml index 83a7d967..d88c9946 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -38,6 +38,10 @@ path = "src/lib.rs" name = "sql-cli" path = "src/main.rs" +[[bin]] +name = "sql-cli-chart" +path = "src/chart_main.rs" + # Windows-specific dependencies @@ -58,6 +62,7 @@ csv = "1.3" ratatui = "0.29" tui-input = { version = "0.11", features = ["crossterm"] } anyhow = "1.0" +clap = "4.0" regex = "1.0" fuzzy-matcher = "0.3" chrono = { version = "0.4", features = ["serde"] } diff --git a/README.md b/README.md index 40c0cc83..09ec0dcf 100644 --- a/README.md +++ b/README.md @@ -364,6 +364,73 @@ FROM products WHERE name.StartsWith('A') ``` +## 📊 Terminal Charts (NEW!) + +SQL CLI now includes a powerful **standalone charting tool** (`sql-cli-chart`) that creates terminal-based visualizations of your SQL query results. Perfect for time series analysis, trend visualization, and data exploration. + +### Chart Tool Usage + +```bash +# Basic time series chart +sql-cli-chart data.csv -q "SELECT time, value FROM data" -x time -y value -t "My Chart" + +# Filter data with SQL WHERE clause +sql-cli-chart trades.csv \ + -q "SELECT timestamp, price FROM trades WHERE symbol = 'AAPL'" \ + -x timestamp -y price -t "AAPL Price Chart" +``` + +### Real-World Example: VWAP Trading Analysis + +Visualize algorithmic trading data with SQL filtering to focus on specific patterns: + +```bash +# Chart fill volume progression for CLIENT orders only +sql-cli-chart data/production_vwap_final.csv \ + -q "SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \ + -x snapshot_time -y filled_quantity \ + -t "CLIENT Order Fill Progression" + +# Compare with ALL orders (shows chaotic "Christmas tree" pattern) +sql-cli-chart data/production_vwap_final.csv \ + -q "SELECT snapshot_time, filled_quantity FROM production_vwap_final" \ + -x snapshot_time -y filled_quantity \ + -t "All Orders - Mixed Pattern" +``` + +**The Power of SQL Filtering**: The first query filters to show only CLIENT orders (991 rows), displaying a clean upward progression. The second shows all 3320 rows including ALGO and SLICE orders, creating a noisy pattern. This demonstrates how SQL queries let you focus on exactly the data patterns you want to visualize. + +### Interactive Chart Controls + +Once the chart opens, use these vim-like controls: +- **hjkl** - Pan left/down/up/right +- **+/-** - Zoom in/out +- **r** - Reset view to auto-fit +- **q/Esc** - Quit + +### Example Scripts + +Ready-to-use chart examples are in the `scripts/` directory: + +```bash +# VWAP average price over time +./scripts/chart-vwap-price.sh + +# Fill volume progression +./scripts/chart-vwap-volume.sh + +# Compare different order types +./scripts/chart-vwap-algo-comparison.sh +``` + +### Chart Features + +- **SQL Query Integration**: Use full SQL power to filter and transform data before charting +- **Smart Auto-Scaling**: Automatically adapts Y-axis range for optimal visibility +- **Time Series Support**: Automatic timestamp parsing and time-based X-axis +- **Interactive Navigation**: Pan and zoom to explore your data +- **Terminal Native**: Pure terminal graphics, no GUI dependencies + ## 🔧 Development ### Running Tests diff --git a/docs/CHARTING_QUICKSTART.md b/docs/CHARTING_QUICKSTART.md new file mode 100644 index 00000000..4119a685 --- /dev/null +++ b/docs/CHARTING_QUICKSTART.md @@ -0,0 +1,137 @@ +# SQL CLI Charting - Quick Start + +## Overview + +SQL CLI now includes a **standalone charting tool** (`sql-cli-chart`) for creating terminal-based visualizations of CSV/JSON data. This is perfect for quick data exploration and analysis without needing external tools. + +## Features + +- **Terminal-native**: Pure ratatui-based charts, no GUI dependencies +- **Time series support**: Automatic timestamp parsing and scaling +- **Interactive controls**: Vim-like navigation (hjkl), pan, zoom +- **Smart auto-scaling**: Adapts Y-axis ranges for optimal data visibility +- **SQL engine integration**: Leverages the full SQL query capabilities + +## Installation + +The chart tool is built as a separate binary alongside the main sql-cli: + +```bash +cargo build --release +# Creates both binaries: +# - ./target/release/sql-cli (main TUI) +# - ./target/release/sql-cli-chart (charting tool) +``` + +## Basic Usage + +```bash +sql-cli-chart -q "SQL_QUERY" -x X_COLUMN -y Y_COLUMN -t "Chart Title" +``` + +### Parameters + +- **file.csv**: Input CSV or JSON file +- **-q, --query**: SQL query to filter/transform data (required) +- **-x, --x-axis**: Column name for X-axis (required) +- **-y, --y-axis**: Column name for Y-axis (required) +- **-t, --title**: Chart title (optional, default: "SQL CLI Chart") +- **-c, --chart-type**: Chart type - line, scatter, bar (optional, default: line) + +## Examples + +### Time Series: VWAP Price Analysis + +```bash +# Basic price over time +./target/release/sql-cli-chart data/production_vwap_final.csv \\ + -q "SELECT snapshot_time, average_price FROM production_vwap_final WHERE filled_quantity > 0" \\ + -x snapshot_time \\ + -y average_price \\ + -t "VWAP Average Price Over Time" + +# Volume analysis +./target/release/sql-cli-chart data/production_vwap_final.csv \\ + -q "SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \\ + -x snapshot_time \\ + -y filled_quantity \\ + -t "Client Order Fill Volume" +``` + +### Scatter Plot: Price vs Volume Correlation + +```bash +./target/release/sql-cli-chart data/production_vwap_final.csv \\ + -q "SELECT average_price, filled_quantity FROM production_vwap_final WHERE filled_quantity > 0" \\ + -x average_price \\ + -y filled_quantity \\ + -c scatter \\ + -t "Price vs Volume Correlation" +``` + +## Interactive Controls + +Once the chart opens, use these vim-like controls: + +- **hjkl**: Pan left/down/up/right +- **+/-**: Zoom in/out +- **r**: Reset view to auto-fit data +- **q/Esc**: Quit + +## Smart Features + +### Auto-Scaling +The tool automatically: +- Detects timestamp columns and converts to epoch time for plotting +- Applies smart Y-axis scaling with 10% padding +- Handles different numeric ranges (prices vs quantities) + +### Time Series Support +Timestamps in RFC3339 format (like `2025-08-12T09:00:18.030000`) are: +- Automatically detected and parsed +- Converted to epoch seconds for plotting +- Displayed as HH:MM:SS on the X-axis + +## Architecture + +The charting system is designed as a **standalone subsystem**: + +- **Non-invasive**: Separate binary, doesn't interfere with main TUI +- **Modular**: Clean separation in `src/chart/` module +- **Extensible**: Easy to add new chart types and features +- **Reusable**: Can be integrated into main TUI later + +## Current Limitations + +1. **No SQL query execution yet**: Currently works with raw data (query parameter ignored for now) +2. **Terminal compatibility**: May have issues in non-TTY environments +3. **Limited chart types**: Only line charts fully implemented + +## Roadmap + +- [ ] Integrate SQL query execution +- [ ] Add candlestick charts for OHLC data +- [ ] Export chart data to CSV +- [ ] Multiple data series on same chart +- [ ] Integration with main TUI (press 'C' for chart mode) + +## Technical Details + +### Data Flow +``` +CSV/JSON → DataTable → DataView → SQL Query → Chart Data → Ratatui Rendering +``` + +### Key Components +- `ChartEngine`: Data processing and query execution +- `LineRenderer`: Time series visualization +- `ChartTui`: Interactive terminal interface +- `ChartViewport`: Pan/zoom state management + +## Troubleshooting + +**"No such device or address" error**: This occurs in environments without proper TTY support. The chart tool requires a real terminal for interactive display. + +**Empty chart**: Check that your SQL query returns data and column names match exactly. + +**Poor scaling**: Use the 'r' key to reset auto-scaling, or manually pan/zoom to find your data. \ No newline at end of file diff --git a/scripts/chart-vwap-algo-comparison.sh b/scripts/chart-vwap-algo-comparison.sh new file mode 100755 index 00000000..f049cb0a --- /dev/null +++ b/scripts/chart-vwap-algo-comparison.sh @@ -0,0 +1,60 @@ +#!/bin/bash + +# Compare ALGO vs CLIENT orders +# This script shows different queries to compare order types + +# Navigate to project root +cd "$(dirname "$0")/.." || exit 1 + +# Check if binary exists +if [ ! -f "./target/release/sql-cli-chart" ]; then + echo "Error: sql-cli-chart binary not found." + echo "Please run 'cargo build --release' first." + exit 1 +fi + +echo "=== VWAP Order Type Comparison ===" +echo "" +echo "Choose which view to display:" +echo "1) CLIENT orders only (clean progression)" +echo "2) ALGO_PARENT orders (algorithm's parent orders)" +echo "3) ALGO_SLICE orders (algorithm's child slices)" +echo "4) All orders (Christmas tree pattern)" +echo "" +read -p "Enter choice (1-4): " choice + +case $choice in + 1) + QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" + TITLE="CLIENT Orders - Fill Progression" + ;; + 2) + QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type = 'ALGO_PARENT'" + TITLE="ALGO Parent Orders - Fill Progression" + ;; + 3) + QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE order_type = 'ALGO_SLICE'" + TITLE="ALGO Slice Orders - Fill Progression" + ;; + 4) + QUERY="SELECT snapshot_time, filled_quantity FROM production_vwap_final WHERE filled_quantity > 0" + TITLE="All Orders - Christmas Tree Pattern" + ;; + *) + echo "Invalid choice. Exiting." + exit 1 + ;; +esac + +echo "" +echo "Executing: $QUERY" +echo "" + +./target/release/sql-cli-chart data/production_vwap_final.csv \ + -q "$QUERY" \ + -x snapshot_time \ + -y filled_quantity \ + -t "$TITLE" + +echo "" +echo "Chart closed." \ No newline at end of file diff --git a/scripts/chart-vwap-price.sh b/scripts/chart-vwap-price.sh new file mode 100755 index 00000000..45b2ec92 --- /dev/null +++ b/scripts/chart-vwap-price.sh @@ -0,0 +1,27 @@ +#!/bin/bash + +# Chart VWAP Average Price Over Time (CLIENT orders only) +# This script visualizes how the VWAP average price evolves for client orders + +# Navigate to project root +cd "$(dirname "$0")/.." || exit 1 + +# Check if binary exists +if [ ! -f "./target/release/sql-cli-chart" ]; then + echo "Error: sql-cli-chart binary not found." + echo "Please run 'cargo build --release' first." + exit 1 +fi + +echo "=== VWAP Average Price Chart ===" +echo "Visualizing CLIENT order average price progression over time" +echo "" + +./target/release/sql-cli-chart data/production_vwap_final.csv \ + -q "SELECT snapshot_time, average_price, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \ + -x snapshot_time \ + -y average_price \ + -t "CLIENT Order VWAP Price Over Time" + +echo "" +echo "Chart closed." \ No newline at end of file diff --git a/scripts/chart-vwap-volume.sh b/scripts/chart-vwap-volume.sh new file mode 100755 index 00000000..6a28f8cf --- /dev/null +++ b/scripts/chart-vwap-volume.sh @@ -0,0 +1,28 @@ +#!/bin/bash + +# Chart VWAP Fill Volume Progression (CLIENT orders only) +# This script visualizes how the filled quantity accumulates for client orders + +# Navigate to project root +cd "$(dirname "$0")/.." || exit 1 + +# Check if binary exists +if [ ! -f "./target/release/sql-cli-chart" ]; then + echo "Error: sql-cli-chart binary not found." + echo "Please run 'cargo build --release' first." + exit 1 +fi + +echo "=== VWAP Fill Volume Chart ===" +echo "Visualizing CLIENT order fill quantity progression" +echo "Expected: Smooth upward trend starting from 0" +echo "" + +./target/release/sql-cli-chart data/production_vwap_final.csv \ + -q "SELECT snapshot_time, average_price, filled_quantity FROM production_vwap_final WHERE order_type LIKE '%CLIENT%'" \ + -x snapshot_time \ + -y filled_quantity \ + -t "CLIENT Order Fill Volume Progression" + +echo "" +echo "Chart closed." \ No newline at end of file diff --git a/src/chart/engine.rs b/src/chart/engine.rs new file mode 100644 index 00000000..6782fdb0 --- /dev/null +++ b/src/chart/engine.rs @@ -0,0 +1,106 @@ +use crate::chart::types::*; +use crate::data::data_view::DataView; +use anyhow::{anyhow, Result}; +use chrono::{DateTime, Utc}; + +pub struct ChartEngine { + data_view: DataView, +} + +impl ChartEngine { + pub fn new(data_view: DataView) -> Self { + Self { data_view } + } + + pub fn execute_chart_query(&mut self, config: &ChartConfig) -> Result { + // The SQL query has already been executed in chart_main.rs + // Just extract chart data from the already-filtered data_view + self.extract_chart_data(&self.data_view, config) + } + + fn extract_chart_data(&self, data: &DataView, config: &ChartConfig) -> Result { + let headers = data.column_names(); + + // Find column indices + let x_col_idx = headers + .iter() + .position(|h| h == &config.x_axis) + .ok_or_else(|| anyhow!("X-axis column '{}' not found", config.x_axis))?; + let y_col_idx = headers + .iter() + .position(|h| h == &config.y_axis) + .ok_or_else(|| anyhow!("Y-axis column '{}' not found", config.y_axis))?; + + let mut points = Vec::new(); + let mut x_min = f64::MAX; + let mut x_max = f64::MIN; + let mut y_min = f64::MAX; + let mut y_max = f64::MIN; + + // Convert data to chart points + for row_idx in 0..data.row_count() { + let x_value_str = data.get_cell_value(row_idx, x_col_idx); + let y_value_str = data.get_cell_value(row_idx, y_col_idx); + + if let (Some(x_str), Some(y_str)) = (x_value_str, y_value_str) { + let (x_float, timestamp) = self.convert_str_to_float_with_time(&x_str)?; + let y_float = self.convert_str_to_float(&y_str)?; + + // Skip invalid data points + if x_float.is_finite() && y_float.is_finite() { + points.push(DataPoint { + x: x_float, + y: y_float, + timestamp, + label: None, + }); + + x_min = x_min.min(x_float); + x_max = x_max.max(x_float); + y_min = y_min.min(y_float); + y_max = y_max.max(y_float); + } + } + } + + if points.is_empty() { + return Err(anyhow!("No valid data points found")); + } + + Ok(DataSeries { + name: format!("{} vs {}", config.y_axis, config.x_axis), + points, + x_range: (x_min, x_max), + y_range: (y_min, y_max), + }) + } + + fn convert_str_to_float_with_time(&self, s: &str) -> Result<(f64, Option>)> { + // Try to parse as timestamp first - handle multiple formats + if let Ok(dt) = DateTime::parse_from_rfc3339(s) { + let utc_dt = dt.with_timezone(&Utc); + // Convert to seconds since epoch for plotting + let timestamp_secs = utc_dt.timestamp() as f64; + Ok((timestamp_secs, Some(utc_dt))) + } else if let Ok(dt) = chrono::NaiveDateTime::parse_from_str(s, "%Y-%m-%dT%H:%M:%S") { + // Handle format like "2025-08-12T09:00:00" (no timezone) + let utc_dt = dt.and_utc(); + let timestamp_secs = utc_dt.timestamp() as f64; + Ok((timestamp_secs, Some(utc_dt))) + } else if let Ok(dt) = chrono::NaiveDateTime::parse_from_str(s, "%Y-%m-%dT%H:%M:%S.%f") { + // Handle format like "2025-08-12T09:00:18.030000" (with microseconds, no timezone) + let utc_dt = dt.and_utc(); + let timestamp_secs = utc_dt.timestamp() as f64; + Ok((timestamp_secs, Some(utc_dt))) + } else if let Ok(f) = s.parse::() { + Ok((f, None)) + } else { + Err(anyhow!("Cannot convert '{}' to numeric value", s)) + } + } + + fn convert_str_to_float(&self, s: &str) -> Result { + s.parse::() + .map_err(|_| anyhow!("Cannot convert '{}' to numeric value", s)) + } +} diff --git a/src/chart/mod.rs b/src/chart/mod.rs new file mode 100644 index 00000000..4ebbcda5 --- /dev/null +++ b/src/chart/mod.rs @@ -0,0 +1,8 @@ +pub mod engine; +pub mod renderers; +pub mod tui; +pub mod types; + +pub use engine::ChartEngine; +pub use tui::ChartTui; +pub use types::*; diff --git a/src/chart/renderers/line.rs b/src/chart/renderers/line.rs new file mode 100644 index 00000000..f4afe1f4 --- /dev/null +++ b/src/chart/renderers/line.rs @@ -0,0 +1,128 @@ +use crate::chart::types::*; +use chrono::{DateTime, Utc}; +use ratatui::{ + layout::Rect, + style::{Color, Style}, + symbols, + widgets::{Axis, Chart, Dataset, GraphType}, + Frame, +}; + +pub struct LineRenderer { + viewport: ChartViewport, +} + +impl LineRenderer { + pub fn new(viewport: ChartViewport) -> Self { + Self { viewport } + } + + pub fn render(&self, frame: &mut Frame, area: Rect, data: &DataSeries, config: &ChartConfig) { + // Convert data points to ratatui format + let chart_data: Vec<(f64, f64)> = data + .points + .iter() + .filter(|p| { + p.x >= self.viewport.x_min + && p.x <= self.viewport.x_max + && p.y >= self.viewport.y_min + && p.y <= self.viewport.y_max + }) + .map(|p| (p.x, p.y)) + .collect(); + + let datasets = self.create_datasets(data, &chart_data); + let (x_axis, y_axis) = self.create_axes(data, config); + + let chart = Chart::new(datasets).x_axis(x_axis).y_axis(y_axis); + + frame.render_widget(chart, area); + } + + fn create_datasets<'a>( + &self, + data: &DataSeries, + chart_data: &'a [(f64, f64)], + ) -> Vec> { + vec![Dataset::default() + .name(data.name.clone()) + .marker(symbols::Marker::Braille) + .style(Style::default().fg(Color::Cyan)) + .graph_type(GraphType::Line) + .data(chart_data)] + } + + fn create_axes(&self, data: &DataSeries, config: &ChartConfig) -> (Axis, Axis) { + // Create X-axis + let x_axis = Axis::default() + .title(config.x_axis.clone()) + .style(Style::default().fg(Color::Gray)) + .bounds([self.viewport.x_min, self.viewport.x_max]) + .labels(self.create_x_labels(data)); + + // Create Y-axis with smart scaling + let y_axis = Axis::default() + .title(config.y_axis.clone()) + .style(Style::default().fg(Color::Gray)) + .bounds([self.viewport.y_min, self.viewport.y_max]) + .labels(self.create_y_labels()); + + (x_axis, y_axis) + } + + fn create_x_labels(&self, data: &DataSeries) -> Vec { + // Check if we have timestamps + let has_timestamps = data.points.iter().any(|p| p.timestamp.is_some()); + + if has_timestamps { + self.create_time_labels() + } else { + self.create_numeric_labels(self.viewport.x_min, self.viewport.x_max) + } + } + + fn create_time_labels(&self) -> Vec { + let num_labels = 5; + let x_span = self.viewport.x_max - self.viewport.x_min; + let step = x_span / (num_labels as f64 - 1.0); + + (0..num_labels) + .map(|i| { + let timestamp_secs = self.viewport.x_min + (i as f64) * step; + let dt = DateTime::::from_timestamp(timestamp_secs as i64, 0) + .unwrap_or_else(|| Utc::now()); + format!("{}", dt.format("%H:%M:%S")) + }) + .collect() + } + + fn create_numeric_labels(&self, min: f64, max: f64) -> Vec { + let num_labels = 5; + let step = (max - min) / (num_labels as f64 - 1.0); + + (0..num_labels) + .map(|i| { + let value = min + (i as f64) * step; + if value.abs() > 1000.0 { + format!("{:.0}k", value / 1000.0) + } else if value.abs() < 1.0 { + format!("{:.3}", value) + } else { + format!("{:.1}", value) + } + }) + .collect() + } + + fn create_y_labels(&self) -> Vec { + self.create_numeric_labels(self.viewport.y_min, self.viewport.y_max) + } + + pub fn viewport_mut(&mut self) -> &mut ChartViewport { + &mut self.viewport + } + + pub fn viewport(&self) -> &ChartViewport { + &self.viewport + } +} diff --git a/src/chart/renderers/mod.rs b/src/chart/renderers/mod.rs new file mode 100644 index 00000000..7ff9f7ae --- /dev/null +++ b/src/chart/renderers/mod.rs @@ -0,0 +1,3 @@ +pub mod line; + +pub use line::LineRenderer; diff --git a/src/chart/tui.rs b/src/chart/tui.rs new file mode 100644 index 00000000..747d5d5f --- /dev/null +++ b/src/chart/tui.rs @@ -0,0 +1,187 @@ +use crate::chart::{engine::ChartEngine, renderers::LineRenderer, types::*}; +use anyhow::Result; +use crossterm::{ + event::{self, Event, KeyCode, KeyEventKind}, + execute, + terminal::{disable_raw_mode, enable_raw_mode, EnterAlternateScreen, LeaveAlternateScreen}, +}; +use ratatui::{ + backend::CrosstermBackend, + layout::{Constraint, Direction, Layout, Margin}, + style::{Color, Style}, + text::{Line, Span}, + widgets::{Block, Borders, Paragraph}, + Terminal, +}; +use std::io; + +pub struct ChartTui { + config: ChartConfig, + data: Option, + renderer: Option, + should_exit: bool, +} + +impl ChartTui { + pub fn new(config: ChartConfig) -> Self { + Self { + config, + data: None, + renderer: None, + should_exit: false, + } + } + + pub fn run(&mut self, mut chart_engine: ChartEngine) -> Result<()> { + // Initialize terminal + enable_raw_mode()?; + let mut stdout = io::stdout(); + execute!(stdout, EnterAlternateScreen)?; + let backend = CrosstermBackend::new(stdout); + let mut terminal = Terminal::new(backend)?; + + // Execute query and prepare data + let data = chart_engine.execute_chart_query(&self.config)?; + let viewport = ChartViewport::new(data.x_range, data.y_range); + let mut renderer = LineRenderer::new(viewport); + renderer.viewport_mut().auto_scale(&data); + + self.data = Some(data); + self.renderer = Some(renderer); + + // Main event loop + let result = self.run_event_loop(&mut terminal); + + // Restore terminal + disable_raw_mode()?; + execute!(terminal.backend_mut(), LeaveAlternateScreen)?; + terminal.show_cursor()?; + + result + } + + fn run_event_loop( + &mut self, + terminal: &mut Terminal>, + ) -> Result<()> { + while !self.should_exit { + terminal.draw(|frame| self.render_chart(frame))?; + + if event::poll(std::time::Duration::from_millis(100))? { + if let Event::Key(key) = event::read()? { + if key.kind == KeyEventKind::Press { + self.handle_key_event(key.code); + } + } + } + } + + Ok(()) + } + + fn render_chart(&mut self, frame: &mut ratatui::Frame) { + let area = frame.area(); + + // Create layout + let chunks = Layout::default() + .direction(Direction::Vertical) + .constraints([ + Constraint::Length(3), // Title + Constraint::Min(0), // Chart + Constraint::Length(3), // Controls + ]) + .split(area); + + // Render title + self.render_title(frame, chunks[0]); + + // Render chart + if let (Some(data), Some(renderer)) = (&self.data, &self.renderer) { + let chart_area = chunks[1].inner(Margin { + horizontal: 1, + vertical: 1, + }); + renderer.render(frame, chart_area, data, &self.config); + } + + // Render controls + self.render_controls(frame, chunks[2]); + } + + fn render_title(&self, frame: &mut ratatui::Frame, area: ratatui::layout::Rect) { + let title = Paragraph::new(Line::from(vec![Span::styled( + &self.config.title, + Style::default().fg(Color::Yellow), + )])) + .block(Block::default().borders(Borders::ALL)); + + frame.render_widget(title, area); + } + + fn render_controls(&self, frame: &mut ratatui::Frame, area: ratatui::layout::Rect) { + let controls = vec![ + Span::raw("Controls: "), + Span::styled("hjkl", Style::default().fg(Color::Cyan)), + Span::raw(" pan • "), + Span::styled("+/-", Style::default().fg(Color::Cyan)), + Span::raw(" zoom • "), + Span::styled("r", Style::default().fg(Color::Cyan)), + Span::raw(" reset • "), + Span::styled("q", Style::default().fg(Color::Cyan)), + Span::raw(" quit"), + ]; + + let paragraph = + Paragraph::new(Line::from(controls)).block(Block::default().borders(Borders::ALL)); + + frame.render_widget(paragraph, area); + } + + fn handle_key_event(&mut self, key_code: KeyCode) { + if let Some(renderer) = &mut self.renderer { + match key_code { + // Exit + KeyCode::Char('q') | KeyCode::Esc => { + self.should_exit = true; + } + + // Pan (vim-style) + KeyCode::Char('h') => { + renderer.viewport_mut().pan(-1.0, 0.0); + } + KeyCode::Char('l') => { + renderer.viewport_mut().pan(1.0, 0.0); + } + KeyCode::Char('k') => { + renderer.viewport_mut().pan(0.0, 1.0); + } + KeyCode::Char('j') => { + renderer.viewport_mut().pan(0.0, -1.0); + } + + // Zoom + KeyCode::Char('+') | KeyCode::Char('=') => { + let viewport = renderer.viewport(); + let center_x = (viewport.x_min + viewport.x_max) / 2.0; + let center_y = (viewport.y_min + viewport.y_max) / 2.0; + renderer.viewport_mut().zoom(1.2, center_x, center_y); + } + KeyCode::Char('-') => { + let viewport = renderer.viewport(); + let center_x = (viewport.x_min + viewport.x_max) / 2.0; + let center_y = (viewport.y_min + viewport.y_max) / 2.0; + renderer.viewport_mut().zoom(0.8, center_x, center_y); + } + + // Reset view + KeyCode::Char('r') => { + if let Some(data) = &self.data { + renderer.viewport_mut().auto_scale(data); + } + } + + _ => {} + } + } + } +} diff --git a/src/chart/types.rs b/src/chart/types.rs new file mode 100644 index 00000000..e57b76d8 --- /dev/null +++ b/src/chart/types.rs @@ -0,0 +1,98 @@ +use chrono::{DateTime, Utc}; +use serde::{Deserialize, Serialize}; + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ChartConfig { + pub title: String, + pub x_axis: String, + pub y_axis: String, + pub chart_type: ChartType, + pub query: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub enum ChartType { + Line, + Scatter, + Bar, + Candlestick, +} + +#[derive(Debug, Clone)] +pub struct DataSeries { + pub name: String, + pub points: Vec, + pub x_range: (f64, f64), + pub y_range: (f64, f64), +} + +#[derive(Debug, Clone)] +pub struct DataPoint { + pub x: f64, + pub y: f64, + pub timestamp: Option>, + pub label: Option, +} + +#[derive(Debug, Clone)] +pub struct ChartViewport { + pub x_min: f64, + pub x_max: f64, + pub y_min: f64, + pub y_max: f64, + pub zoom_level: f64, +} + +impl ChartViewport { + pub fn new(x_range: (f64, f64), y_range: (f64, f64)) -> Self { + Self { + x_min: x_range.0, + x_max: x_range.1, + y_min: y_range.0, + y_max: y_range.1, + zoom_level: 1.0, + } + } + + pub fn auto_scale(&mut self, data: &DataSeries) { + self.x_min = data.x_range.0; + self.x_max = data.x_range.1; + + // Smart Y-axis scaling with padding + let y_span = data.y_range.1 - data.y_range.0; + let padding = y_span * 0.1; // 10% padding + self.y_min = data.y_range.0 - padding; + self.y_max = data.y_range.1 + padding; + } + + pub fn pan(&mut self, dx: f64, dy: f64) { + let x_span = self.x_max - self.x_min; + let y_span = self.y_max - self.y_min; + + self.x_min += dx * x_span * 0.1; + self.x_max += dx * x_span * 0.1; + self.y_min += dy * y_span * 0.1; + self.y_max += dy * y_span * 0.1; + } + + pub fn zoom(&mut self, factor: f64, center_x: f64, center_y: f64) { + let x_span = self.x_max - self.x_min; + let y_span = self.y_max - self.y_min; + + let new_x_span = x_span / factor; + let new_y_span = y_span / factor; + + self.x_min = center_x - new_x_span / 2.0; + self.x_max = center_x + new_x_span / 2.0; + self.y_min = center_y - new_y_span / 2.0; + self.y_max = center_y + new_y_span / 2.0; + + self.zoom_level *= factor; + } +} + +impl Default for ChartType { + fn default() -> Self { + ChartType::Line + } +} diff --git a/src/chart_main.rs b/src/chart_main.rs new file mode 100644 index 00000000..31cabc7d --- /dev/null +++ b/src/chart_main.rs @@ -0,0 +1,155 @@ +use anyhow::Result; +use clap::{Arg, Command}; +use sql_cli::{ + chart::types::{ChartConfig, ChartType}, + chart::{ChartEngine, ChartTui}, + data::data_view::DataView, + data::datatable_loaders::{load_csv_to_datatable, load_json_to_datatable}, + data::query_engine::QueryEngine, +}; +use std::path::Path; + +fn main() -> Result<()> { + let matches = Command::new("sql-cli-chart") + .version("1.34.0") + .about("Standalone charting tool for CSV/JSON data with SQL queries") + .arg( + Arg::new("file") + .help("Input CSV or JSON file") + .required(true) + .index(1), + ) + .arg( + Arg::new("query") + .short('q') + .long("query") + .value_name("SQL") + .help("SQL query to filter/transform data") + .required(true), + ) + .arg( + Arg::new("x-axis") + .short('x') + .long("x-axis") + .value_name("COLUMN") + .help("Column for X-axis") + .required(true), + ) + .arg( + Arg::new("y-axis") + .short('y') + .long("y-axis") + .value_name("COLUMN") + .help("Column for Y-axis") + .required(true), + ) + .arg( + Arg::new("title") + .short('t') + .long("title") + .value_name("TITLE") + .help("Chart title") + .default_value("SQL CLI Chart"), + ) + .arg( + Arg::new("chart-type") + .short('c') + .long("chart-type") + .value_name("TYPE") + .help("Chart type (line, scatter, bar)") + .default_value("line"), + ) + .get_matches(); + + // Parse arguments + let file_path = matches.get_one::("file").unwrap(); + let query = matches.get_one::("query").unwrap(); + let x_axis = matches.get_one::("x-axis").unwrap(); + let y_axis = matches.get_one::("y-axis").unwrap(); + let title = matches.get_one::("title").unwrap(); + let chart_type_str = matches.get_one::("chart-type").unwrap(); + + // Parse chart type + let chart_type = match chart_type_str.to_lowercase().as_str() { + "line" => ChartType::Line, + "scatter" => ChartType::Scatter, + "bar" => ChartType::Bar, + "candlestick" => ChartType::Candlestick, + _ => { + eprintln!( + "Invalid chart type '{}'. Supported types: line, scatter, bar, candlestick", + chart_type_str + ); + std::process::exit(1); + } + }; + + // Load data + println!("Loading data from '{}'...", file_path); + let data_view = load_data_file(file_path)?; + println!( + "Loaded {} rows, {} columns", + data_view.row_count(), + data_view.column_count() + ); + + // Execute the SQL query to get filtered data + println!("Executing query: {}", query); + let query_engine = QueryEngine::new(); + let filtered_view = + query_engine.execute(std::sync::Arc::new(data_view.source().clone()), query)?; + println!( + "Query result: {} rows, {} columns", + filtered_view.row_count(), + filtered_view.column_count() + ); + + // Create chart configuration (query already executed) + let config = ChartConfig { + title: title.clone(), + x_axis: x_axis.clone(), + y_axis: y_axis.clone(), + chart_type, + query: query.clone(), + }; + + // Create chart engine with filtered data + let chart_engine = ChartEngine::new(filtered_view); + let mut chart_tui = ChartTui::new(config); + + println!("Starting chart TUI... Press 'q' to quit"); + + // Run the chart TUI + chart_tui.run(chart_engine)?; + + println!("Chart closed."); + Ok(()) +} + +fn load_data_file(file_path: &str) -> Result { + let path = Path::new(file_path); + + if !path.exists() { + return Err(anyhow::anyhow!("File '{}' does not exist", file_path)); + } + + let extension = path + .extension() + .and_then(|ext| ext.to_str()) + .unwrap_or("") + .to_lowercase(); + + let data_table = match extension.as_str() { + "csv" => load_csv_to_datatable(file_path, "chart_data")?, + "json" => load_json_to_datatable(file_path, "chart_data")?, + _ => { + return Err(anyhow::anyhow!( + "Unsupported file format '{}'. Supported formats: csv, json", + extension + )); + } + }; + + // Create DataView from DataTable + Ok(DataView::new(std::sync::Arc::new(data_table))) +} diff --git a/src/lib.rs b/src/lib.rs index bc2883cd..9663bb43 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -1,5 +1,6 @@ // New module structure (gradually moving files here) pub mod api; +pub mod chart; pub mod config; pub mod core; pub mod data; diff --git a/src/non_interactive.rs b/src/non_interactive.rs index 062eb0c3..ab213f6f 100644 --- a/src/non_interactive.rs +++ b/src/non_interactive.rs @@ -8,7 +8,6 @@ use tracing::{debug, info}; use crate::data::data_view::DataView; use crate::data::datatable::{DataTable, DataValue}; use crate::data::datatable_loaders::{load_csv_to_datatable, load_json_to_datatable}; -use crate::data::query_engine::QueryEngine; use crate::services::query_execution_service::QueryExecutionService; /// Output format for query results diff --git a/test-chart-simple.sh b/test-chart-simple.sh new file mode 100755 index 00000000..475c72ef --- /dev/null +++ b/test-chart-simple.sh @@ -0,0 +1,26 @@ +#!/bin/bash + +echo "=== Testing Chart Tool Data Processing ===" + +echo "1. Testing CSV loading and basic data processing..." +echo " File: data/production_vwap_final.csv" +echo " Expected: Should load 3320 rows, 20 columns successfully" +echo "" + +# Test basic data loading (will fail at TUI step but show data loading works) +echo "Running chart tool (will exit at TUI due to no TTY):" +./target/release/sql-cli-chart data/production_vwap_final.csv \ + -q "SELECT snapshot_time, average_price FROM production_vwap_final WHERE filled_quantity > 0" \ + -x snapshot_time \ + -y average_price \ + -t "VWAP Price Over Time" + +echo "" +echo "=== Success Indicators ===" +echo "✅ Data loading works if you see: 'Loaded 3320 rows, 20 columns'" +echo "✅ Chart processing works if you see: 'Starting chart TUI...'" +echo "✅ TTY error is expected in this environment" +echo "" +echo "=== Next Steps ===" +echo "To test interactively, run in a proper terminal:" +echo "./target/release/sql-cli-chart data/production_vwap_final.csv -q \"SELECT snapshot_time, average_price FROM production_vwap_final WHERE filled_quantity > 0\" -x snapshot_time -y average_price -t \"VWAP Price Over Time\"" \ No newline at end of file