diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 00000000..00ad71fb --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,3 @@ +{ + "typescript.tsdk": "node_modules\\typescript\\lib" +} \ No newline at end of file diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md new file mode 100644 index 00000000..8e3a34f4 --- /dev/null +++ b/DOCUMENTATION.md @@ -0,0 +1,76 @@ +# Kine-Sight Documentation + +## Overview +**Kine-Sight** is an interactive, browser-based AI fitness coaching application. It leverages real-time local AI processing to track the user's fitness movements, count repetitions, verify correct posture using visual pose estimation, and provide real-time motivating and corrective feedback through a local Large Language Model (LLM). + +Because it relies on WebAssembly (WASM) and browser-based AI inference, it is highly private and fast—processing happens entirely on the user's device without needing external API calls to remote servers. + +## Tech Stack & Dependencies +- **Frontend Framework**: React 19 + TypeScript +- **Bundler**: Vite +- **Computer Vision (Pose Detection)**: `@mediapipe/tasks-vision` runs MediaPipe's lightweight pose-landmarker model in the browser. +- **On-Device LLM (Voice/Text Coach)**: `@mlc-ai/web-llm` enables running local LLMs (e.g., Llama-3-8B-Instruct) natively in the browser using WebGPU and WebAssembly. +- **Audio Routing**: Web Audio API manages immediate correct/incorrect rep sound indications. + +--- + +## Core Architecture and Data Flow + +### 1. `src/App.tsx` & `src/main.tsx` +These act as the entry points of the application. `App.tsx` serves as a simple shell that mounts the primary view: `FitnessTab`. + +### 2. `src/components/FitnessTab.tsx` +This is the core view and primary state-machine controller for the application. + +**Key responsibilities:** +- **UI Architecture**: Displays the pre-workout "Dashboard/Bento" view with exercise choices. During a workout, it renders the video feed, the canvas overlay (for the skeleton), and the AI Coach's rolling text feedback. +- **Lifecycle Management**: + - Activates the camera (`getUserMedia`). + - Initializes the MediaPipe Pose model. + - Initializes the WebLLM coach in the background. + - Coordinates a pre-workout countdown. +- **Detection Loop (`startDetectionLoop`)**: + - Runs recursively via `requestAnimationFrame` to sample frames from the active `