Oscar — ORTHOGONAL STOCHASTIC CONTROL FOR ALIGNMENT-RESPECTING DIVERSITY IN FLOW MATCHING

Jingxuan Wu*<sup>1 </sup>, Zhenglin Wan*<sup>2 </sup>, Xingrui Yu <sup>3 </sup>, Yuzhe Yang <sup>4 </sup>, Bo An <sup>1 </sup>, Ivor Tsang <sup>3 </sup><br>

<sup>1 </sup>Department of Statistics and Operations Research, UNC-Chapel Hill, America <sup>2 </sup>College of Computing and Data Science, NTU, Singapore <sup>3 </sup>Centre for Frontier AI Research, A*STAR, Singapore <sup>4 </sup> UCSB

(*: Equal contribution)

Official implementation of our paper: OSCAR: ORTHOGONAL STOCHASTIC CONTROL FOR ALIGNMENT-RESPECTING DIVERSITY IN FLOW MATCHING arXiv: https://arxiv.org/abs/2510.09060

This repository provides the official codebase for Oscar. It includes:

• scripts/ — three baselines (DPP, CADS, PG) and our method for SD3/SD3.5 (Diffusers).
• experiments/ — experiment drivers used to reproduce the results in our paper.
• eval/ — unified evaluation tools (coverage/diversity/quality) and CSV aggregation.
• Reference pointers for DIM/CIM diversity evaluation (see: https://github.com/facebookresearch/DIMCIM).

---

Overview

Figure — Our guidance mechanism: OSCAR applies orthogonal stochastic control to steer flow-matching trajectories for diversity while respecting alignment.

---

Table of Contents

• Installation
• Models & Checkpoints
• Quick Start
• DIM/CIM Reference Evaluation
• Citing
• License

---

Installation

git clone https://github.com/Johnny221B/OSCAR
cd OSCAR
python -m venv .venv && source .venv/bin/activate
pip install -U pip
pip install -e .

Models & Checkpoints

• Use local Diffusers SD3 / SD3.5 pipelines. Example layout: models/stable-diffusion-3.5-medium/.Model page: Stable Diffusion 3.5 Medium — Hugging Face.
• Loaders auto-resolve nested directories by searching for model_index.json.
• OpenAI CLIP (JIT). Provide ViT-B-32.pt locally (e.g., ~/.cache/clip/ViT-B-32.pt). If absent, the code falls back to open_clip.

Quick Start

    CUDA_VISIBLE_DEVICES=0,1 python -u scripts/ourmethod.py \
      --spec specs/multi_class.json \
      --guidances 3.0 \
      --seeds 1111 2222 3333 4444 \
      --model-dir ./models/stable-diffusion-3.5-medium \
      --clip-jit ~/.cache/clip/ViT-B-32.pt

DIM/CIM Reference Evaluation

For DIM/CIM (diversity) evaluation, please follow the official repository:https://github.com/facebookresearch/DIMCIM

Typical workflow:

1. Generate images with this repo (by concept/prompt/guidance/seed).
2. Use DIM/CIM’s evaluation code to compute their diversity metrics.
3. Optionally align our CSV outputs for side-by-side comparisons.

Citing

If you find this repository useful, please consider to cite:

@misc{wu2025oscarorthogonalstochasticcontrol,
      title={OSCAR: Orthogonal Stochastic Control for Alignment-Respecting Diversity in Flow Matching}, 
      author={Jingxuan Wu and Zhenglin Wan and Xingrui Yu and Yuzhe Yang and Bo An and Ivor Tsang},
      year={2025},
      eprint={2510.09060},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2510.09060}, 
}

License

This project is released under the MIT License.See the LICENSE file for the full text.

Optional (recommended): add an SPDX tag to source files:

SPDX-License-Identifier: MIT