Skip to content

CodeRabbit: "Fix" Catch-up-math #486

@AlexanderLavelle

Description

@AlexanderLavelle

From #484 --

-        for _ in range(
-            self.start_window,
-            (loaded_checkpoint_window + 1) * self.hparams.inner_steps,
-        ):
-            self.inner_scheduler.step()
+        # Advance scheduler by the exact number of inner steps since the start window
+        steps_to_apply = max(
+            0,
+            (loaded_checkpoint_window - self.start_window + 1)
+            * self.hparams.inner_steps,
+        )
+        for _ in range(steps_to_apply):
+            self.inner_scheduler.step()
self.sync_window = loaded_checkpoint_window

Code rabbit explains that one over-steps the LR scheduler by start_window * (inner_steps-1)

This is likely negligible but we should still modify the math to achieve the desired state

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions