Skip to content

refactor: refined sparse reward environment#119

Closed
neuralsorcerer wants to merge 8 commits intofacebookresearch:mainfrom
neuralsorcerer:refactor-sre
Closed

refactor: refined sparse reward environment#119
neuralsorcerer wants to merge 8 commits intofacebookresearch:mainfrom
neuralsorcerer:refactor-sre

Conversation

@neuralsorcerer
Copy link
Contributor

@neuralsorcerer neuralsorcerer commented Jul 13, 2025

Changes:

  • Rewrote SparseRewardEnvironment.observation_space to expose a 4‑dimensional box for agent and goal coordinates and added deterministic seeding on reset.
  • Updated continuous‐action step logic to return float tensors, apply time–limit truncation only when the goal is not reached, and defined a bounded BoxActionSpace.
  • Improved the discrete environment and validated the action index before converting to the continuous step, keeping predefined action vectors.
  • Added tests for ContinuousSparseRewardEnvironment.

Why these changes?

  • The observation space previously described only agent coordinates now returning the full agent‑goal state clarifies environment outputs and matches tests.
  • The continuous environment now exposes a usable action space and handles termination/truncation correctly, improving compatibility with RL algorithms.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 13, 2025
@facebook-github-bot
Copy link
Contributor

@Yonathae has imported this pull request. If you are a Meta employee, you can view this in D78899735.

@neuralsorcerer neuralsorcerer closed this by deleting the head repository Sep 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants