Collaborative On-Sensor Array Cameras

Collaborative On-Sensor Array Cameras
Jipeng Sun¹, Kaixuan Wei², Thomas Eboli⁴, Congli Wang¹, Cheng Zheng¹, Zhihao Zhou³, Arka Majumdar³, Wolfgang Heidrich², Felix Heide^1
1 Princeton University, ² KAUST, ³ University of Washington ⁴ Université Paris-Saclay

ACM Transactions on Graphics (SIGGRAPH), 2025

This work proposes a collaboratively optimized 2×3 metasurface array (100M+ nanoposts) that enables on-sensor, video-rate broadband imaging with a 3.6 mm form factor and a non-generative, parallax-aware reconstruction pipeline.

Repository Timeline

• [08/2025] Environment setup & repo skeleton
• [08/2025] Paper, supplementary, and project page links
• [08/2025] Release synthetic hyperspectral dataset and collaborative reconstrution
• [09/2025] Release experimental dataset

Setup

Start by cloning the repository

git clone https://github.com/princeton-computational-imaging/collab-array-cameras.git
cd collab-array-cameras

Conda Setup

# create conda environment
conda env create -f environment.yml
conda activate collab-array

Demo Run

# simply run demo.py
python demo.py

Dataset

Please find the experimental capture dataset here: Experimental Dataset .

The dataset contains 653 indoor and outdoor experimental captured scenes. Each scene contains the raw sensor measurement and cropped six sublenses' captures of the metalens array camera. Each scene also contains a homography matrix for pair cameras alignment.

Citation

Please consider citing our work if you make use of this codebase.

@article{collab_array,
  author    = {Sun, Jipeng and Wei, Kaixuan and Eboli, Thomas and Wang, Congli and
               Zheng, Cheng and Zhou, Zhihao and Majumdar, Arka and Heidrich, Wolfgang
               and Heide, Felix},
  title     = {Collaborative On-Sensor Array Cameras},
  journal   = {ACM Trans. Graph.},
  volume    = {44},
  number    = {4},
  articleno = {55},
  numpages  = {18},
  year      = {2025},
  month     = {jul},
  publisher = {Association for Computing Machinery},
  doi       = {10.1145/3731200},
  keywords  = {computational optics, computational imaging, distributed optimization}
}