[v0.2.5+] Support Types

[m96-chan]

Supported Type Combinations

Input A	Input B	Accumulate	Output	Status	Notes
FP32	FP32	FP32	FP32	✅ Supported	Standard CUDA FMA path
TF32	TF32	FP32	FP32	✅ Supported	TensorCore (Ampere+), matmul only
FP16	FP16	FP32	FP16	🔜 Planned	TensorCore, mixed precision
FP16	FP16	FP32	FP32	🔜 Planned	Common for training/inference
BF16	BF16	FP32	BF16	🔜 Planned	Preferred for modern LLMs
BF16	BF16	FP32	FP32	🔜 Planned	Higher numerical stability
FP16	FP16	FP16	FP16	❌ Out of scope (v0.x)	Low stability, niche use
INT8	INT8	INT32	FP32	❌ Out of scope (v0.x)	Requires calibration & quantization
Mixed (FP16/BF16/FP32)	Mixed	FP32	FP32	❌ Not yet	Explicit casts required

---

Current Implementation Status (v0.2.4)

Operation	FP32	TF32	FP16	BF16
add	✅	—	🔜	🔜
mul	✅	—	🔜	🔜
matmul	✅ (18 TFLOPS)	✅ (27.38 TFLOPS)	🔜	🔜

---

Design Notes

• FP32 and TF32 are the baseline and already supported.
• FP16 and BF16 with FP32 accumulation are considered mandatory for practical ML workloads and are planned next.
• Automatic mixed precision (AMP), loss scaling, and quantized INT8 execution are intentionally excluded from v0.x to keep the API simple and predictable.
• Explicit cast() operations are preferred over implicit type promotion, following NumPy-style semantics.

---

Scope Clarification (Non-goals for v0.x)

• Automatic AMP
• INT8 / quantized kernels
• Training-specific features (loss scaling, stochastic rounding)

These may be considered after the runtime becomes fully CUDA Toolkit–independent and the core API stabilizes.

---

Related Issues

• [v0.2.5+] Support Operator #59 Support Operator (operation coverage)
• [v1.0] TF32 Kernel Further Optimization (CUTLASS-level) #53 TF32 Kernel Further Optimization

[m96-chan]

FP16/BF16 Support Implementation Complete

FP16 (float16) and BF16 (bfloat16) support has been implemented for all core operators.

Implementation Summary

C++ Native Backend:

• Added Float16 and BFloat16 to DataType enum in native/core/types.hpp
• Implemented FP16/BF16 kernels for all elementwise operations (add, sub, mul, div)
• Implemented FP16/BF16 kernels for all unary operations (exp, log, relu)
• Implemented FP16/BF16 kernels for all reduction operations (sum, mean, max)
• Implemented FP16/BF16 matmul in native/ops/matmul_f16_bf16.cuh
• All operations use FP32 accumulation for numerical stability

Python API:

• Added float16 and bfloat16 DataType objects in pygpukit.core.dtypes
• Updated GPUArray._wrap_native() to handle Float16/BFloat16
• Updated validation functions to allow FP16/BF16 for float operations
• Exported float16 and bfloat16 in pygpukit.__init__

Pybind11 Bindings:

• Added Float16/BFloat16 to DataType enum export
• Updated from_numpy() to detect np.float16 and np.uint16 (for BF16)
• Updated to_numpy() to create appropriate output arrays

Updated Status Table

Operation	FP32	TF32	FP16	BF16
add	✅	—	✅	✅
sub	✅	—	✅	✅
mul	✅	—	✅	✅
div	✅	—	✅	✅
exp	✅	—	✅	✅
log	✅	—	✅	✅
relu	✅	—	✅	✅
sum	✅	—	✅	✅
mean	✅	—	✅	✅
max	✅	—	✅	✅
matmul	✅ (18 TFLOPS)	✅ (27.38 TFLOPS)	✅	✅

Test Results

All 166 existing tests pass. Comprehensive FP16/BF16 tests (22 additional tests) all pass:

• FP16: add, sub, mul, div, exp, log, relu, sum, mean, max, matmul
• BF16: add, sub, mul, div, exp, log, relu, sum, mean, max, matmul

Notes

• FP16/BF16 matmul uses simple kernels with FP32 accumulation for correctness
• TensorCore WMMA optimization for FP16/BF16 matmul can be added in future for better performance
• BF16 uses uint16 storage in NumPy (no native bfloat16 type)

[m96-chan]

Completed in v0.2.5

FP16/BF16 Data Types

• Added gpk.float16 and gpk.bfloat16 dtypes
• FP32 accumulation for numerical stability
• astype() method for dtype conversion

Elementwise Operations

All elementwise ops now support FP16/BF16:

• add, sub, mul, div
• Operator overloads: +, -, *, /

Matrix Multiplication

• FP16 matmul with FP32 accumulation
• BF16 matmul with FP32 accumulation

Benchmark Results (RTX 3090 Ti)

Size	FP16	BF16
2048²	2.4 TFLOPS	2.4 TFLOPS
4096²	2.4 TFLOPS	2.3 TFLOPS
8192²	2.3 TFLOPS	2.3 TFLOPS

Note: TensorCore optimization for FP16/BF16 tracked in #60

PR: #61