When there is only one eye being drawn, don't add the per-eye offset based on the IPD. Just draw from head space.