Monado 3D Display — Unity Plugin

Unity plugin for rendering on eye-tracked 3D light field displays via the Monado OpenXR runtime. Works with any OpenXR-compatible 3D display.

Overview

The plugin intercepts Unity's OpenXR pipeline at the native layer to provide:

Eye-tracked stereo rendering — Kooima asymmetric frustum projection from real-time eye positions
Two stereo rig modes — Camera-centric (add to existing camera) or display-centric (place a virtual display in the scene)
2D UI overlay — Route any Canvas to a window-space composition layer with stereo disparity
Editor preview — Side-by-side stereo preview without leaving the editor

The plugin works by hooking xrLocateViews before Unity sees the results, replacing the runtime's FOV data with Kooima-computed asymmetric frustums. Unity then builds correct projection matrices through its normal rendering pipeline — no Camera.SetStereoProjectionMatrix hacks required.

Requirements

Requirement	Details
Unity	2022.3 LTS or later (including Unity 6)
OpenXR Plugin	`com.unity.xr.openxr` 1.9.1+ (installed via Package Manager)
XR Plugin Management	`com.unity.xr.management` 4.4.0+ (auto-installed with OpenXR)
Monado Runtime	Pre-built from openxr-3d-display CI — see Deploying to End Users

Platform Support

Editor Platform	Build Target	Native Plugin	Status
Windows	Windows x64	`monado3d_unity.dll`	Supported
macOS	macOS	`libmonado3d_unity.dylib`	Supported
macOS	Windows x64	`monado3d_unity.dll`	Supported (cross-compile)

Installing the Plugin

Option A: From Local Folder (Development)

Clone and build the native plugin, then add the package from disk:

git clone https://github.com/dfattal/unity-3d-display.git
cd unity-3d-display/native~
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release    # on Windows, add: -A x64
cmake --build . --config Release

In Unity: Window > Package Manager > + > Add package from disk... → select unity-3d-display/package.json.

Option B: From Git URL (after a release)

Note: The upm branch is created by CI when a v* tag is pushed. If no release has been published yet, use Option A.

In Unity: Window > Package Manager
Click + > Add package from git URL...

Enter:

https://github.com/dfattal/unity-3d-display.git#upm

This installs from the upm branch which includes pre-built native binaries. To pin a specific version:

https://github.com/dfattal/unity-3d-display.git#upm/v0.1.0

Option C: From Release Tarball

Download the .tgz file from the latest release
In Unity: Window > Package Manager
Click + > Add package from tarball...
Select the downloaded .tgz file

After installation, the package appears as Monado 3D Display in the Package Manager.

Enabling the Feature

Go to Edit > Project Settings > XR Plug-in Management
Under the Standalone tab (Windows or macOS), check OpenXR
Click the gear icon next to OpenXR (or expand OpenXR > Features)
Enable Monado 3D Display

You can verify the runtime connection under Project Settings > XR Plug-in Management > OpenXR > Monado 3D Display — a status panel shows whether XR_RUNTIME_JSON is set.

Scene Setup

Camera-Centric Mode (Recommended)

This is the simplest setup. Your existing Main Camera stays in place and becomes the nominal viewing position. The plugin creates stereo eye offsets around it.

Step 1: Select your Main Camera in the hierarchy.

Step 2: Add Component > Monado3DCamera (found under Monado3D category).

That's it. The component:

Reads eye tracking data from the runtime each frame
Computes Kooima asymmetric frustums around the camera position
Pushes modified FOVs into the OpenXR pipeline before Unity renders

Hierarchy:
  Main Camera              <-- has Camera + Monado3DCamera
    ├── Scene objects...
    └── (everything else in your scene)

Inspector tunables:

Parameter	Default	Description
IPD Factor	1.0	Scales inter-eye distance. <1 = reduced stereo, >1 = exaggerated
Parallax Factor	1.0	Scales eye X/Y offset from viewing axis
Convergence Distance	auto	Distance to the virtual screen plane (meters). Auto-set from display info if 0.
Field of View	auto	Computed from convergence + display size. Override with non-zero value (degrees).

When to use camera-centric mode:

First-person games and apps
Retrofitting stereo into an existing project (just add the component)
When the camera moves freely through the scene

Display-Centric Mode

The display is a fixed object in the scene. Eye positions are transformed relative to this virtual display.

Step 1: Create an empty GameObject where the virtual display should be:

GameObject > Create Empty, name it "VirtualDisplay"
Position and orient it in the scene (e.g., at (0, 1.5, 2) facing the player)

Step 2: Add Component > Monado3DDisplay

Step 3: Make your Main Camera a child of this object, or position it separately — the display's world transform is sent to the native plugin as the "scene transform" applied to raw eye positions.

Hierarchy:
  VirtualDisplay           <-- has Monado3DDisplay
    └── Main Camera        <-- standard Camera component

Inspector tunables:

Parameter	Default	Description
IPD Factor	1.0	Scales inter-eye distance
Parallax Factor	1.0	Scales eye X/Y offset from display center
Perspective Factor	1.0	Scales eye Z only (depth intensity without changing baseline)
Scale Factor	1.0	Virtual display size relative to physical display (affects Kooima screen extents)

When to use display-centric mode:

Digital signage, museum exhibits, kiosks
The virtual display is a physical object in the scene (e.g., a TV on a wall)
You want the display's position and orientation to be an explicit scene element

Try it: Import the Display Scene sample from Package Manager for a ready-made tabletop turntable demo.

Stereo Tunables Reference

Both modes share a common tunable interface that maps to the native plugin's Kooima computation. Here's what each parameter does physically:

IPD Factor (both modes)

Scales the horizontal distance between left and right eye positions.

1.0 = natural inter-pupillary distance from eye tracker
0.5 = half the real IPD (gentler stereo, less eye strain)
2.0 = double the real IPD (exaggerated depth, "macro" effect)
0.0 = mono rendering (both eyes at center)

Parallax Factor (both modes)

Scales the eye's X and Y offset from the viewing axis (or display center).

1.0 = natural head parallax
0.0 = no parallax (stereo still works via IPD, but no motion parallax)
>1.0 = exaggerated motion parallax

Perspective Factor (display-centric only)

Scales the eye's Z position (distance from display) without changing the baseline.

1.0 = natural depth
<1.0 = eyes appear closer to display (stronger perspective)
>1.0 = eyes appear farther (flatter perspective)

Scale Factor (display-centric only)

Scales the virtual display dimensions used in the Kooima projection.

1.0 = virtual display matches physical display size
2.0 = virtual display twice as large (scene appears smaller)
0.5 = virtual display half-size (scene appears larger)

Convergence Distance (camera-centric only)

Distance from the camera to the virtual screen plane, in meters. Objects at this distance appear at the display surface; closer objects pop out, farther objects recede.

Default: auto-set from the display's nominalViewerZ (~0.5m for typical displays)
Increase for deeper scenes, decrease for tabletop/near-field content

Field of View (camera-centric only)

Vertical FOV override in degrees. When 0, auto-computed from convergence distance and physical display height. Override when you need a specific FOV regardless of display geometry.

2D UI Overlay

Route a Canvas to a window-space composition layer that the Monado compositor overlays on both eyes with per-eye disparity shift (renders pre-interlace).

Create a Canvas (any render mode)
Add Component > Monado3DWindowSpaceUI to the Canvas
Configure position and size in fractional window coordinates [0..1]:

Parameter	Default	Description
Position X	0.0	Left edge of overlay in window (0 = left, 1 = right)
Position Y	0.0	Bottom edge (0 = bottom, 1 = top)
Width	1.0	Fractional width
Height	1.0	Fractional height
Disparity	0.0	Stereo disparity in pixels. 0 = at screen plane, positive = in front
Resolution	512	Render texture resolution (square)

The overlay is submitted as XrCompositionLayerWindowSpaceEXT and composited by Monado before display processing. This means 2D UI text stays sharp and is not interlaced — ideal for HUDs, menus, and status displays.

Editor Preview

Side-by-Side Preview (no runtime needed)

Window > Monado3D > Preview Window
Select SideBySide mode
The window shows a stereo pair computed from your scene cameras

This works without the Monado runtime running — useful for authoring and layout.

Runtime Readback Preview (actual display processing)

Add a Monado3DPreview component to the same GameObject as your camera
In the Preview Window, select RuntimeReadback mode
The preview shows the actual composited + display-processed output

Requires Monado runtime running. Use this for final QA verification.

Building Your App

Windows Standalone

File > Build Settings
Select Windows, Mac, Linux platform
Set Target Platform to Windows and Architecture to x86_64
Verify in Player Settings > XR Plug-in Management > Standalone that OpenXR is enabled with the Monado 3D Display feature
Click Build or Build And Run

The build output includes monado3d_unity.dll in the Plugins/ folder alongside your executable.

macOS Standalone

File > Build Settings
Select macOS platform
Verify OpenXR + Monado 3D Display feature enabled in Standalone XR settings
Click Build

The .app bundle includes libmonado3d_unity.dylib in the plugins folder.

Cross-Compiling (macOS Editor to Windows Build)

Yes, you can build a Windows app from Unity for Mac. Unity's cross-compilation support handles this:

Install the Windows Build Support module in Unity Hub:
- Unity Hub > Installs > your Unity version > Add Modules > Windows Build Support (Mono)
In Unity: File > Build Settings > Windows, Mac, Linux
Set Target Platform to Windows
Build as normal

The plugin includes both platform binaries (monado3d_unity.dll for Windows, libmonado3d_unity.dylib for macOS). Unity automatically selects the correct one based on the build target.

Important: The built Windows .exe still requires the Monado runtime installed on the target Windows machine — see Deploying to End Users. You cannot run the Windows build on macOS.

Deploying to End Users

Your built app is a standard OpenXR application. It needs an OpenXR runtime on the target machine. The plugin is hardware-agnostic — it communicates only through the OpenXR API and does not depend on any specific display vendor SDK.

Windows Deployment

Install the Monado runtime via the SRMonadoInstaller.exe from the openxr-3d-display CI build artifact (registers the runtime JSON and copies DLLs system-wide).

Or, for development/testing, set the environment variable:

set XR_RUNTIME_JSON=C:\path\to\openxr_monado-dev.json

macOS Deployment

Set before launching the app:

export XR_RUNTIME_JSON=/path/to/SRMonado-macOS/share/openxr/1/openxr_monado.json

What Happens Without the Runtime

If the Monado runtime is not installed, Unity's OpenXR loader fails to find a runtime and logs:

[OpenXR] No OpenXR runtime found

The Monado3DFeature logs a warning but doesn't crash — your app runs in mono (non-stereo) mode. You can check Monado3DFeature.Instance being null to detect this and show a user-facing message.

Testing Without Hardware

The sim_display driver provides a software 3D display for development:

# Windows
set SIM_DISPLAY_ENABLE=1
set SIM_DISPLAY_OUTPUT=sbs          # side-by-side stereo
# or: anaglyph (red-cyan), blend (50/50 alpha)

# macOS
export SIM_DISPLAY_ENABLE=1
export SIM_DISPLAY_OUTPUT=sbs

With sim_display:

Eye tracking is simulated via keyboard/mouse (qwerty driver)
Display dimensions are synthetic (configurable)
Output renders as SBS, anaglyph, or alpha-blend in a regular window

This lets you develop and test the full stereo pipeline on any machine.

Troubleshooting

Symptom	Cause	Fix
"No OpenXR runtime found"	`XR_RUNTIME_JSON` not set or points to missing file	Set the env var to the Monado runtime JSON path
Black screen	Monado 3D Display feature not enabled	Check Project Settings > XR Plug-in Management > OpenXR > Features
No stereo (flat image)	Eye tracking not running	Verify the Monado runtime is configured with a display that supports eye tracking, or use sim_display for testing
Stereo looks wrong	Tunables misconfigured	Reset to defaults (IPD=1, Parallax=1, Scale=1)
`DllNotFoundException: monado3d_unity`	Native plugin not found by Unity	Ensure the plugin binaries are in `Runtime/Plugins/Windows/x64/` or `Runtime/Plugins/macOS/`
HDRP stereo artifacts	Single-pass instanced issue	Verify both eye views have correct FOVs in Frame Debugger
Editor preview blank	No preview component or wrong mode	Add Monado3DPreview component; use SideBySide mode without runtime
`VK_ERROR_EXTENSION_NOT_PRESENT` on macOS	MoltenVK limitation	Known issue — use sim_display for testing

Debug Logging

Enable Log Eye Tracking on the Monado3DCamera or Monado3DDisplay component to see per-frame eye positions in the Console:

[Monado3D] Eyes: L=(0.032, 0.001, 0.504), R=(-0.031, 0.001, 0.504), tracked=True

Checking Runtime Status in Editor

Project Settings > XR Plug-in Management > OpenXR > Monado 3D Display shows:

Runtime JSON path and whether the file exists
Runtime connection status
Connected display properties (resolution, physical size, nominal viewer distance)
Eye tracking status

Architecture

Unity Editor / Player
┌──────────────────────────────────────────────────────────┐
│  C# Layer                                                │
│                                                          │
│  Monado3DFeature : OpenXRFeature                         │
│    HookGetInstanceProcAddr → install native hooks        │
│    OnSystemChange → query XrDisplayInfoEXT               │
│    API: SetTunables(), SetSceneTransform()               │
│                                                          │
│  Monado3DCamera          Monado3DDisplay                 │
│  (camera-centric)        (display-centric)               │
│  Attach to Camera        Place in scene                  │
│                                                          │
│  Monado3DWindowSpaceUI   Monado3DPreview                 │
│  (2D overlay)            (editor preview)                │
└──────────────────────────────────────────────────────────┘
        │ P/Invoke (monado3d_unity.dll / .dylib)
        ▼
┌──────────────────────────────────────────────────────────┐
│  Native Plugin (C/C++)                                   │
│  Hook chain:                                             │
│    xrLocateViews → scene transform → tunables → Kooima  │
│    xrCreateSession → inject window binding               │
│    xrGetSystemProperties → extract display info          │
│    xrEndFrame → submit overlay layers                    │
│  Thread-safe double-buffered shared state                │
└──────────────────────────────────────────────────────────┘
        │ Standard OpenXR API
        ▼
┌──────────────────────────────────────────────────────────┐
│  Monado Runtime                                          │
│  Compositor → Display Processor → Output                 │
└──────────────────────────────────────────────────────────┘

Transform Chain (per frame)

Eye Tracker → raw positions in DISPLAY space
    ↓
Scene Transform (parent camera pose, zoom)
    ↓
Tunables (IPD factor, parallax, perspective, scale)
    ↓
Kooima Asymmetric Frustum → XrFovf angles
    ↓
Unity builds projection matrices → renders stereo

Package Contents

Path	Purpose
`Runtime/Monado3DFeature.cs`	OpenXR Feature — lifecycle hooks, P/Invoke, public API
`Runtime/Monado3DCamera.cs`	Camera-centric stereo rig MonoBehaviour
`Runtime/Monado3DDisplay.cs`	Display-centric stereo rig MonoBehaviour
`Runtime/Monado3DDisplayInfo.cs`	Display properties data struct
`Runtime/Monado3DTunables.cs`	Tunable parameters struct
`Runtime/Monado3DWindowSpaceUI.cs`	2D UI overlay routing
`Runtime/Monado3DPreview.cs`	Editor preview (SBS + readback)
`Runtime/Monado3DNative.cs`	P/Invoke bindings to native plugin
`Runtime/Plugins/Windows/x64/`	Windows native plugin (DLL)
`Runtime/Plugins/macOS/`	macOS native plugin (dylib)
`Editor/Monado3DDisplayEditor.cs`	Custom inspector for display-centric mode
`Editor/Monado3DCameraEditor.cs`	Custom inspector for camera-centric mode
`Editor/Monado3DPreviewWindow.cs`	Editor preview window
`Editor/Monado3DSettingsProvider.cs`	Project Settings page
`native~/`	Native C/C++ plugin source + CMakeLists.txt

Name		Name	Last commit message	Last commit date
Latest commit History 13 Commits
.claude/skills/ci-monitor		.claude/skills/ci-monitor
.github/workflows		.github/workflows
Editor		Editor
Runtime		Runtime
Samples~		Samples~
docs		docs
native~		native~
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
Editor.meta		Editor.meta
LICENSE		LICENSE
README.md		README.md
Runtime.meta		Runtime.meta
package.json		package.json

Folders and files

Latest commit

History

Repository files navigation

Monado 3D Display — Unity Plugin

Table of Contents

Overview

Requirements

Platform Support

Installing the Plugin

Option A: From Local Folder (Development)

Option B: From Git URL (after a release)

Option C: From Release Tarball

Enabling the Feature

Scene Setup

Camera-Centric Mode (Recommended)

Display-Centric Mode

Stereo Tunables Reference

IPD Factor (both modes)

Parallax Factor (both modes)

Perspective Factor (display-centric only)

Scale Factor (display-centric only)

Convergence Distance (camera-centric only)

Field of View (camera-centric only)

2D UI Overlay

Editor Preview

Side-by-Side Preview (no runtime needed)

Runtime Readback Preview (actual display processing)

Building Your App

Windows Standalone

macOS Standalone

Cross-Compiling (macOS Editor to Windows Build)

Deploying to End Users

Windows Deployment

macOS Deployment

What Happens Without the Runtime

Testing Without Hardware

Troubleshooting

Debug Logging

Checking Runtime Status in Editor

Architecture

Transform Chain (per frame)

Package Contents

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages