Epic: Measure and improve FIW kinship verification model

[jasoneplumb]

Overview

The current Siamese CNN scores an estimated AUC of ~0.50-0.55 (random guessing). Top competition entries achieve 0.85-0.92. This epic tracks the prioritized improvements to close that gap.

Current State

• 2-layer CNN trained from scratch on 2,000 of ~257K available pairs
• 56×56 input resolution, contrastive loss, no augmentation
• Critical bugs: inverted loss labels, .count method bug, Windows-only paths
• No evaluation metrics beyond raw loss

Sub-Issues (priority order)

#	Issue	Type	Est. AUC Impact
1	#2 — Fix contrastive loss label convention	bug	+0.05-0.15
2	#3 — Fix code bugs (.count, paths, seeds)	bug	correctness
3	#4 — Use full training dataset	enhancement	+0.10-0.15
4	#5 — Remove prediction rounding	bug	+0.05-0.10
5	#6 — Add evaluation metrics (AUC, ROC)	enhancement	measurement
6	#7 — Pretrained backbone (FaceNet/ArcFace)	enhancement	+0.20-0.30
7	#8 — Resolution + data augmentation	enhancement	+0.05-0.10
8	#9 — Semver versioning + changelog	enhancement	project hygiene

Suggested Order

1. Fix bugs first (Fix contrastive loss label convention #2, Fix code bugs: .count method, path separators, reproducibility #3, Remove prediction rounding — output full float precision #5) — trivial effort, correctness-critical
2. Add metrics (Add evaluation metrics: AUC, accuracy, ROC curves #6) — need measurement before optimization
3. Use full dataset (Use full training dataset instead of 2K subsample #4) — trivial code change, large impact
4. Pretrained backbone (feat: Replace CNN backbone with pretrained face model (FaceNet/ArcFace) #7) — biggest single AUC gain
5. Resolution + augmentation (feat: Increase input resolution and add data augmentation #8) — complements the backbone swap
6. Update version + changelog after each milestone (feat: Add semver versioning and changelog #9)

Success Criteria

• AUC ≥ 0.80 on a family-aware validation split
• All code bugs fixed, reproducible results (seeded RNG)

[jasoneplumb]

Status Update — 2026-03-18

All 8 sub-issues are now closed:

#	Issue	Status
#2	Fix contrastive loss labels	✅ Done (v0.2.0)
#3	Fix code bugs	✅ Done (v0.3.0)
#4	Use full training dataset	✅ Done (v0.5.0)
#5	Remove prediction rounding	✅ Done (v0.3.0)
#6	Add evaluation metrics	✅ Done (v0.4.0)
#7	Pretrained FaceNet backbone	✅ Done (v0.6.0)
#8	Resolution + augmentation	✅ Done (v0.6.0)

Remaining: Training run in progress on CPU (~60 hrs ETA). Epic will be closed once AUC ≥ 0.80 is confirmed on the family-aware validation split. A new Kaggle submission will be generated from the trained model.