Skip to content

Add noise to combat exposure bias#214

Draft
maharajamihir wants to merge 11 commits intomainfrom
add-noise-to-combat-exposure-bias
Draft

Add noise to combat exposure bias#214
maharajamihir wants to merge 11 commits intomainfrom
add-noise-to-combat-exposure-bias

Conversation

@maharajamihir
Copy link

No description provided.

)
noise_BTNM = jax.random.normal(_rng_noise, shape=(B, T, N, M))
noise_bucket_idx_B = jnp.floor(
(noise_level_B / self.max_noise_level) * self.noise_buckets

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is the culprit for our new nans, add a small epsilon in the denominator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants