Skip to content

Overview

Jump to bottom
Kyle Drake edited this page Oct 3, 2025 · 1 revision

Overview

Abstract

This project implements a fully customizable in-vehicle computing environment built on Arch Linux running on the NVIDIA Jetson Orin Nano 8GB. Unlike conventional head unit integrations that limit functionality to vendor-locked ecosystems, this platform directly projects a custom Linux interface to the vehicle’s infotainment system via USB device emulation. The system consolidates multiple advanced subsystems — radar signal processing, LIDAR/laser detection, vehicular telemetry, multi-camera vision, and on-device machine learning — into a unified, extensible framework. The goal is to create a research-grade automotive platform capable of both real-time situational awareness and autonomous reasoning.

Motivation

Modern vehicles increasingly integrate proprietary infotainment and ADAS (advanced driver assistance system) features, but these remain opaque black boxes. Enthusiasts, researchers, and engineers lack the ability to extend or analyze the system beyond consumer-grade APIs. This project seeks to:

  • Provide full user sovereignty over the in-car computing stack.
  • Bridge high-performance edge AI with automotive sensors.
  • Enable modular experimentation in DSP, AI inference, and vehicular UX.
  • Offer a transparent alternative to closed OEM ecosystems.

System Architecture

The platform can be decomposed into four major layers:

1. Hardware Layer

  • Jetson Orin Nano 8GB: primary compute module (ARM CPU + Ampere GPU + Tensor cores).
  • Software-defined radios (SDRs): wideband capture of RF emissions for radar detection.
  • Downconverters + LNAs + filters: frequency translation for X, K, and Ka radar bands.
  • Optical sensors: laser/LIDAR detection modules for 905 nm pulses.
  • OBD telemetry: Bluetooth or serial integration with OBDLink MX+.
  • Cameras: USB and CSI multi-camera feeds for situational awareness.
  • GPS via iPhone hotspot: NMEA feed tunneled over Wi-Fi hotspot connection to Jetson, enabling positioning without a dedicated GPS module.
  • IMU: optional inertial sensors for dead-reckoning and stabilization.

2. OS & Projection Layer

  • Arch Linux: chosen for its minimalism and reproducibility.
  • Custom Interface: a KDE Plasma or Wayland-based compositor, stripped and rebuilt for driving use.
  • USB gadget emulation: Jetson (via a Pi Zero 2 W or dedicated UDC front-end) presents itself as an Android Auto device, allowing the custom interface to project to the factory head unit.

3. Signal & Data Processing Layer

  • GNU Radio / SoapySDR: real-time spectral analysis, CFAR detection, and POP burst detection.
  • DSP pipelines: optimized for Jetson GPU/Tensor cores to accelerate FFT and filtering.
  • Sensor fusion: cross-correlation of radar, laser, GPS (from hotspot), IMU, and OBD streams.
  • Logging subsystem: NVMe-backed storage with timestamped event persistence.

4. AI & Autonomy Layer

  • On-device LLMs: 7B–13B quantized models running under llama.cpp or equivalent.
  • Task orchestration: LLM-driven agent layer for contextual reasoning (alerts, advice, self-reflection).
  • Adaptive dashboard: dynamically restructures UI layouts and warnings based on current situational data.
  • Autonomy research hooks: sandboxed environment for experimenting with semi-autonomous decision support.

Key Research Challenges

  1. Real-time RF Analysis: Detecting short-lived radar bursts (POP) under constraints of sampling rate, bandwidth, and noise.
  2. Multi-modal Sensor Fusion: Aligning RF, optical, OBD, camera, and iPhone GPS streams to a common temporal base.
  3. Edge AI Constraints: Running sufficiently large LLMs within 8 GB RAM and limited power envelope.
  4. Interface Design: Balancing user control with safety, minimal latency, and distraction reduction.
  5. Connectivity Integration: Reliably ingesting GPS and network data over iPhone hotspot while maintaining low-latency DSP.

Applications

  • Automotive radar and LIDAR characterization.
  • Prototyping next-generation driver assistance systems.
  • Experimentation with edge AI inference in dynamic environments.
  • Creation of sovereign in-car operating systems independent of OEM control.
  • Educational use in DSP, embedded systems, vehicular networking, and sensor fusion.

Roadmap (High-Level)

  1. Baseline projection: Arch + custom desktop to head unit.
  2. K-band integration: RFbeam module with Jetson FFT pipeline.
  3. Multi-band support: add X and Ka downconverters + calibration.
  4. Laser sensing: photodiode array integration.
  5. GPS hotspot integration: finalize NMEA parsing from iPhone Wi-Fi feed.
  6. Full sensor fusion: unify radar, laser, OBD, GPS, and cameras.
  7. AI overlay: integrate local LLM assistant with real-time event reasoning.
  8. Dashboard UX refinement: design a usable, safe, extensible UI.

Conclusion

This project reimagines the vehicle infotainment system as an open research platform: a testbed for advanced DSP, AI, and human-machine interfaces. By combining Arch Linux minimalism, Jetson GPU acceleration, SDR front-ends, and GPS data sourced directly from a phone hotspot, it enables end-to-end sovereignty over in-car computing — from raw RF photons to high-level AI reasoning.

Next: Hardware Requirements

Clone this wiki locally