V2V Collaborative Perception Pipeline

Semantic Segmentation + XOR Delta Encoding + Autonomous Decision Making for Urban Driving Safety
HACK4IMPACT Track 2 | Team Lead: Hridam Biswas | KIIT University

---

The Problem

A single vehicle cannot see around corners, behind trucks, or into occluded zones. Vehicle-to-Vehicle (V2V) perception shares segmentation masks between nearby vehicles so each car can reason about hazards it cannot directly observe.

This pipeline implements a three-stage approach:

Vehicle A captures frame
        │
        ▼
[Part 1] DeepLabV3 Segmentation → Binary mask → SQLite
        │
        ▼
[Part 2] XOR Delta + 3×3 Grid Zone Detection
        │
        ▼
[Part 3] 48K-param CNN → Driving Decision + Alert

---

Setup

git clone https://github.com/Hridambiswas/Hack4IMPACTTrack2-404_Error_Found.git
cd Hack4IMPACTTrack2-404_Error_Found
pip install -r requirements.txt

---

Part 1 — Segment & Store

MobileNetV3-based DeepLabV3 segments an input frame. Binary mask, class labels, and frame metadata are stored in SQLite for retrieval by downstream stages.

python part1_segment_store.py --image frame1.jpg --frame_id 001  # reference (clear road)
python part1_segment_store.py --image frame2.jpg --frame_id 002  # new observation

DB Schema — segmentation_masks

Column	Type	Description
frame_id	VARCHAR	Unique frame identifier
timestamp	TIMESTAMP	UTC capture time
shape	INTEGER[]	Mask dimensions [H, W]
mask	BYTEA	Binary segmentation mask
class_labels	JSONB	Detected object classes

---

Part 2 — XOR Delta + Grid Zone Detection

Loads two masks from the DB, computes their XOR difference, and maps changed pixels to a spatial 3×3 grid to localize hazards.

python part2_xor_zones.py --frame_a 001 --frame_b 002

┌──────────┬──────────┬──────────┐
│ TOP-LEFT │  TOP-CTR │ TOP-RGT  │
├──────────┼──────────┼──────────┤
│  MID-LFT │  CENTER  │  MID-RGT │
├──────────┼──────────┼──────────┤
│ BOT-LEFT │  BOT-CTR │ BOT-RGT  │
└──────────┴──────────┴──────────┘

[ pedestrian ]  zone=BOTTOM-CENTER   Δpx=312
[ car        ]  zone=TOP-RIGHT       Δpx=87

---

Part 3 — Tiny Decision CNN

~48K parameter CNN classifies the XOR delta patch and outputs a structured driving decision with urgency scoring.

# Train (requires data_dir with class subfolders: pedestrian, vehicle, cyclist, clear)
python part3_decision_model.py --mode train --data_dir ./cityscapes_masks --epochs 20

# Inference
python part3_decision_model.py --mode infer --frame_a 001 --frame_b 002

Output:

=======================================================
  AUTONOMOUS DECISION OUTPUT
=======================================================
  STOP — Pedestrian detected
  Zone      : BOTTOM-CENTER
  Distance  : 23.0m
  Urgency   : HIGH
  Confidence: 91.3%
=======================================================

Urgency Thresholds:

Class	CRITICAL	HIGH	MEDIUM	LOW
Pedestrian	<15m	<30m	<60m	>60m
Cyclist	<20m	<40m	<80m	>80m
Vehicle	<10m	<25m	<60m	>60m

No training data? A rule-based fallback using only the zone report from Part 2 is included — no model required for demo purposes.

---

End-to-End Pipeline

from part1_segment_store import run as segment_and_store
from part3_decision_model import run_inference

segment_and_store("frame_clear.jpg",           "001")
segment_and_store("frame_with_pedestrian.jpg", "002")

result = run_inference("001", "002")
# → {"alert": "STOP — Pedestrian detected", "zone": "BOTTOM-CENTER",
#    "distance": "23.0m", "urgency": "HIGH", "confidence": "91.3%"}

---

Project Structure

├── part1_segment_store.py   # DeepLabV3 segmentation + SQLite storage
├── part2_xor_zones.py       # XOR delta computation + grid zone mapping
├── part3_decision_model.py  # Decision CNN training + inference + rule fallback
└── requirements.txt

---

Author

Hridam Biswas — IEEE Researcher, KIIT University
GitHub · Portfolio