LaRa (Low-Cost Adaptive Robotic-AI Therapy System) is an emotion-aware AI therapy companion built for two intersecting populations:
Primary: Neurodiverse children and young adults — including those with autism spectrum disorder, Down syndrome, ADHD, cerebral palsy, and related neurodevelopmental conditions — where structured, predictable, emotion-aware interaction supports learning, communication, and development.
Secondary: Individuals with progressive or acquired neurological conditions — including frontotemporal dementia (Pick's disease), Parkinson's disease, traumatic brain injury, and aphasia — where consistent, gentle, adaptive engagement supports cognitive maintenance, emotional regulation, and quality of life. These conditions are often incurable, but structured therapeutic interaction demonstrably improves daily functioning and wellbeing.
LaRa does not replace clinical therapy. It extends therapeutic reach by providing consistent, safe, structured engagement between formal sessions.
LaRa is designed to:
- Detect user presence using real-time face detection.
- Estimate emotional state through a vision-based emotion recognition model.
- Track engagement levels and emotional trends over time.
- Select appropriate therapeutic response categories using deterministic logic.
- Generate context-aware speech and visual guidance using a constrained language model.
- Log session data for caregiver and therapist review.
The system prioritizes structured control and safety over unrestricted AI autonomy.
LaRa is organized into four major layers:
- Captures real-time video input.
- Performs face detection to confirm user presence.
- Estimates emotional state using valence, arousal, and confidence metrics.
- Applies temporal smoothing to stabilize emotion outputs.
- Estimates engagement level.
- Detects emotional trends (improving, declining, stable).
- Identifies frustration indicators.
This layer ensures that decisions are based on stable behavioral context rather than frame-level fluctuations.
- Uses deterministic rule-based logic.
- Selects response categories such as:
- Encourage
- Simplify
- Progress
- Pause
- End Session
- Enforces safety thresholds and engagement constraints.
Behavioral decisions are governed exclusively by this module.
- Receives structured contextual state.
- Generates speech and visual suggestions aligned with the selected response category.
- Operates under strict output constraints.
- Does not influence decision-making.
This separation ensures adaptive communication while maintaining predictable system behavior.
- Sensor Input (Camera / Microphone)
- Preprocessing (Frame normalization and synchronization)
- Face Detection (Presence validation)
- Emotion Estimation (Valence, Arousal, Confidence)
- State Aggregation (Trend detection and engagement scoring)
- Decision Engine (Response category selection)
- Language Generation (Context-aware speech rendering)
- Response Execution (Text-to-Speech and visual cues)
- Logging and Session Summary Generation
When a session begins, LaRa activates visual input and verifies user presence. If a face is detected, emotional signals are extracted and converted into stable internal state representations.
The decision engine evaluates engagement and emotional trends to determine the appropriate therapeutic response type. The language generation module produces supportive and context-aware speech aligned with that decision.
If disengagement persists, the system pauses or ends the session. All interactions are logged to generate structured summaries for therapists and caregivers.
LaRa functions as a structured, adaptive engagement assistant designed for safe and explainable interaction.
- Improved engagement and learning outcomes for neurodiverse children.
- Enhanced cognitive maintenance and quality of life for individuals with neurological conditions.
- Emotion-aware adaptive interaction tailored to diverse cognitive needs.
- Objective session monitoring for therapists, caregivers, and family members.
- Safe, explainable, and scalable system behavior.
LaRa demonstrates a structured integration of perception models and language models within a safety-bounded robotic therapy support system.
We have successfully developed the core foundational layers of LaRa, achieving significant milestones:
- Core Engine: Developed a robust Perception, Decision, and Generation engine that allows LaRa to perceive and respond deterministically to emotions.
- Persistent Runtime Storage: Refactored the architecture to ensure secure, local persistence of session artifacts, vector stores (RAG), and SQLite memory outside of the Git repository, preserving continuity across therapeutic sessions.
- Real-time Pipeline Optimization: Optimized the AI pipeline for speed and efficiency, strictly adhering to constraints critical for neurodiverse interactions—such as maintaining gentle pacing, high predictability, and structured engagement.
- Clinical Dashboard Overhaul: Completely redesigned the Tier 2 Clinical Dashboard UI, transforming it from an AI research interface into a clean, light-themed, professional hospital UI designed for therapists to monitor interactions.
Our immediate priorities to refine the LaRa system include:
- Dashboard Enhancements: Refining the Tier 2 Clinical Dashboard to provide therapists with structured metric cards, subtle trend tracking, and actionable insights.
- Latency Optimization: Conducting continuous optimization of the real-time interaction loop to ensure completely natural and fluid conversational pacing.
- Memory Logic Refinement: Improving the underlying memory retrieval logic (
AgentricTLMandlara_vector_store) to enable deeper, more personalized, long-term therapeutic engagement.
Looking ahead, we are expanding LaRa's capabilities to maximize its therapeutic impact:
- Physical Embodiment: Integrating the software core with full physical robotic hardware to transition from a digital assistant to a tangible companion.
- Advanced RAG Architecture: Expanding the memory architecture for sophisticated long-term personalization, allowing LaRa to remember distinct preferences and interaction history across months and years.
- Multilingual Support: Adding comprehensive multilingual capabilities to serve diverse demographics and classroom environments.
- HIS Integration: Developing secure API extensions for seamless integration with existing Hospital Information Systems and electronic health records.
LaRa is positioned to redefine therapeutic support by:
- Democratizing Access: Establishing a highly scalable, low-cost therapeutic tool that makes quality engagement available in budget-constrained classrooms, care facilities, and homes.
- Pioneering Ethical AI: Leading the development of ethical, deterministic-AI hybrid systems for neurodiverse care that prioritize safety over unconstrained AI autonomy.
- Empowering Caregivers: Significantly reducing therapist and caregiver workload by acting as an asynchronous engagement assistant, extending therapeutic support beyond the conventional clinic.
# 1. Clone the repository
git clone https://github.com/deepanmpc/LaRa.git
cd LaRa
# 2. Create a virtual environment and install dependencies
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# 3. Download all required AI models (one-time setup)
python scripts/download_models.py
# 4. Start LaRa
python src/main.pyNote: Models are stored in
runtime/models/and reused across sessions. SetLARA_DATA_DIRto override the runtime path for cluster deployments..