Solution Validation Program

[alyxya]

Can code be released that validates a solution conforms to the constraints of the problem? I feel that this would be the best way to resolve ambiguities and any confusion about the problem.

[yarongmu-google]

Thanks for the suggestion. To clarify the philosophy of the challenge:

1. Optimality: we cannot provide a tool that checks if a solution is "good" or "optimal," as finding the best schedule is the core objective of the competition.

2. Constraint validation: regarding a tool to check if a solution is merely valid (e.g., fits in memory, compatible granularity):

   • We do not plan to release a standalone validation tool at this stage.

   • We view "self-validation" - ensuring your solution adheres to the specification - as a core part of engineering.

   • Just like in real-world system design, there is often no external "test harness" provided upfront. You must translate the constraints in PROBLEM.md into your own internal checks and that's part of the challenge.

We recommend building your own lightweight "sanity checker" functions based on the rules in the problem description.

[yarongmu-google]

I will resolve this for now. Please reopen if you still think we should turn this problem from "design by contract" to "guess and check".

[alyxya]

I think this is the wrong philosophy because the real world has physical limitations and rules that are absolute, whereas words on a document often contain mistakes and ambiguities. If the specs were instead in the form of code, it would be unambiguous with expected behavior because there's only one interpretation. Engineering is about ensuring things conform to the source of truth, so it's more worthwhile to ensure things work on real hardware vs a simulated hardware, or the compiler's output matches expectations vs whatever the documentation said the compiler should do. If I found a conflict between the hardware's documented specs and its real behavior, I would have to conform to the real behavior. The real world has GPUs and compilers to run and verify machine learning code on, which is an unambiguous real world validation test.

The current setup is more like guessing and checking to me (except without the checking part) where I have too many confusions regarding my mental model of the setup and what is allowed or not and what I should be imagining the hardware is doing. Having some kind of underlying hardware model or validation program in the form of code means I don't have to make any guesses and I can treat that as the design contract.

I don't have permissions to reopen this issue.