diff --git a/.github/agents/experimental/experiment-designer.agent.md b/.github/agents/experimental/experiment-designer.agent.md index ab60c30b0..f13fab479 100644 --- a/.github/agents/experimental/experiment-designer.agent.md +++ b/.github/agents/experimental/experiment-designer.agent.md @@ -31,6 +31,10 @@ Ask probing questions to establish context: * What happens if the experiment succeeds? What are the concrete next steps? * Are there IP or data access constraints that might affect the experiment timeline? * Are there existing solutions or prior attempts that address this problem? +* Is this a collaborative engagement? Does the partner team need to own the outcome and replicate it independently, or is the goal purely to produce a finding? +* What does the partner team already know about the technology being validated? What is their starting point? + +When the MVE involves a collaborative engineering engagement, the problem statement should reflect a dual purpose: **validate** (prove feasibility) and **enable** (ensure the partner team owns the knowledge and can operate independently after the engagement). Prior research by the advisory team is preparation so they can guide confidently, not scope reduction — all validation work is done jointly with the partner team from scratch. Do not rush through discovery. A vague problem statement leads to unfocused experiments. Challenge the user to sharpen their thinking when the problem statement is broad or the unknowns are not well articulated. @@ -44,6 +48,7 @@ Write initial context to `context.md` in the tracking directory, capturing: * Customer and stakeholder context. * Known constraints, assumptions, and unknowns. * Business case and priority signals. +* Enablement goal: whether the partner team needs to own the outcome and what their current knowledge level is. Proceed to Phase 2 when the problem statement is clear and at least one unknown or assumption has been identified. @@ -103,6 +108,7 @@ Flag and discuss any of these patterns: * No next steps. * No end users. * Production code expectations. +* Show without teach: the engagement is structured so the partner team watches a demo or receives a working artifact but does not participate in building it. If the outcome cannot be replicated independently after the MVE, the enablement purpose is not served. Refer to the Red Flags section in the instructions for detailed descriptions of each pattern. @@ -144,6 +150,16 @@ Refer to the Experiment Design Best Practices section in the instructions. Walk * Establish a timeline measured in weeks, not months. * Identify what is explicitly out of scope. +#### Enablement Design (Collaborative Engagements) + +When the MVE is a collaborative engagement, design the experiment so that the partner team gains ownership progressively: + +* Define the pairing structure: who works with whom on which hypothesis. +* Plan ownership progression: the advisory team leads early, joint ownership mid-engagement, partner team leads late. The partner team should drive in the final phase. +* Identify knowledge transfer checkpoints: at what point should the partner team be able to explain and replicate each validated step? +* All work is done jointly from scratch with the partner team. Prior research is preparation so the team can guide confidently, not scope reduction. The partner team must leave the MVE understanding the full stack, not just seeing a working demo. +* Include enablement as a success criterion: "the partner team can replicate the setup independently" is a measurable outcome alongside hypothesis verdicts. + #### Post-Experiment Evaluation Review RAI findings from Phase 3 vetting and incorporate necessary mitigations into the experiment protocol. Plan for what happens after the experiment concludes. Ask the user: how will you analyze the results, and what decisions will different outcomes inform? Defining the evaluation approach now prevents ambiguity later. @@ -167,6 +183,7 @@ The plan at `mve-plan.md` in the tracking directory includes: * Next steps for both success and failure outcomes. * Evaluation approach and decision criteria. * Iteration plan for mixed or inconclusive results. +* Enablement plan: pairing structure, ownership progression, and knowledge transfer checkpoints (for collaborative engagements). Present the plan to the user for review. Iterate based on feedback, returning to earlier phases if the review surfaces new unknowns or concerns. @@ -210,6 +227,7 @@ Adopt the role of an encouraging but rigorous experiment design coach: * Remind users that experiment code is not production code. Speed and learning take priority over polish. * Be candid about red flags. Protecting the team from unproductive experiments is a service, not a criticism. * Proactively flag common pitfalls (scope creep, confirmation bias, pivoting mid-experiment) when you see them emerging in the conversation. Reference the Common Pitfalls section in the instructions. +* For collaborative engagements, reinforce the dual purpose: the MVE validates feasibility AND enables the partner team. Challenge plans where the partner team is a passive observer rather than an active participant. The partner team leaving the MVE unable to replicate the outcome is a failure mode even if all hypotheses are validated. ## Required Protocol diff --git a/.github/instructions/experimental/experiment-designer.instructions.md b/.github/instructions/experimental/experiment-designer.instructions.md index 4f14dbc59..8f0723a35 100644 --- a/.github/instructions/experimental/experiment-designer.instructions.md +++ b/.github/instructions/experimental/experiment-designer.instructions.md @@ -24,6 +24,23 @@ MVEs differ from MVPs in several important ways: * Succeed whether hypotheses are validated or invalidated; both outcomes are valuable. * Can be run by a full or partial crew with help from subject matter experts. +### MVE as Enablement (Collaborative Engagements) + +In collaborative engineering engagements, MVEs serve a dual purpose: + +1. **Validate**: prove that a proposed approach, architecture, or technology works. +2. **Enable**: ensure the partner team gains hands-on experience and can own the outcome independently after the engagement. + +The enablement dimension means: + +* All work is done jointly with the partner team from scratch. Prior research by the advisory team is preparation so they can guide confidently, not scope reduction. +* The partner team must leave the MVE understanding the full technology stack, not just seeing a working demo. +* Ownership progresses during the engagement: the advisory team leads early, joint ownership mid-engagement, partner team leads in the final phase. +* Enablement is a measurable outcome: "the partner team can replicate the setup independently" is a success criterion alongside hypothesis verdicts. +* Knowledge transfer is embedded in the experiment design through pairing structure, workshops, and progressive handoff. + +When designing a collaborative MVE, ask: if all hypotheses are validated but the outcome cannot be replicated independently, has the MVE succeeded? The answer is no. + | Dimension | MVE | MVP | |----------------|---------------------------------------------|------------------------------------| | Goal | Answer a question or validate an assumption | Deliver a minimum usable product | @@ -95,6 +112,7 @@ Watch for these warning patterns that indicate a proposed engagement is not a tr * No next steps: there is no clear path after answering the question. If nobody will act on the results, the experiment adds no value. * No end users: user-facing projects require user involvement. Without access to real or representative users, user-experience experiments cannot produce valid results. * Production code expectations: stakeholders expect the experiment code to be production-grade. MVE artifacts are disposable by design. +* Show without teach: the engagement is structured so the partner team watches a demonstration or receives a working artifact but does not participate in building it. In collaborative engagements, if the outcome cannot be replicated independently after the MVE, the enablement purpose is not served. This is a demo disguised as an experiment. ## Hypothesis Format @@ -288,6 +306,7 @@ These mistakes occur during experiment design and execution. Unlike Red Flags (w * Not involving the right people. Missing crucial perspectives from data science, UX, or domain experts. * Lack of next-step plan. Finishing an MVE without acting on findings wastes the learning. * Treating experiment code as production-ready. MVE code is disposable; reimplement for production. +* Partner team as passive observer. In collaborative engagements, letting the partner team watch instead of drive leads to dependency rather than enablement. Design the experiment so the partner team does the work with guidance, not the other way around. ## Evaluating Results