Skip to content

feat(agents): add LLM Council agent and councilor subagents#1327

Open
andrewDoing wants to merge 1 commit intomicrosoft:mainfrom
andrewDoing:anvineyard/llm-council
Open

feat(agents): add LLM Council agent and councilor subagents#1327
andrewDoing wants to merge 1 commit intomicrosoft:mainfrom
andrewDoing:anvineyard/llm-council

Conversation

@andrewDoing
Copy link
Copy Markdown
Contributor

Pull Request

Description

Adds an LLM Council agent that dispatches the same question package to GPT-5.4, Opus 4.6, and Gemini 3.1 Pro councilor subagents, then synthesizes consensus, disagreement, assumptions, uncertainty, and a recommended next step into one user-facing answer.

This contribution is scoped as a reusable decision-support pattern for hve-core rather than a broader orchestration platform. It keeps the first upstream cut focused on agent artifacts and discoverability updates.

Related Issue(s)

Closes #1326

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Compare two implementation approaches and tell me where the models agree, where they disagree, and what we should do next.

Execution Flow:

  1. The LLM Council agent frames one shared question package.
  2. The same package is sent to GPT-5.4, Opus 4.6, and Gemini 3.1 Pro councilor subagents.
  3. Each councilor answers independently.
  4. The LLM Council agent synthesizes consensus, disagreement, uncertainty, and a recommended next step.

Output Artifacts:

The agent returns one synthesized response rather than three separate answers. The response is expected to include a direct answer, consensus points, disagreement and uncertainty, model-by-model summaries, and a final recommendation.

Success Indicators:

The response clearly distinguishes agreement from disagreement, identifies uncertainty, and ends with an actionable recommendation instead of a raw transcript dump.

Testing

Validation has not been run yet in this PR creation flow.

Recommended validation before merge:

  • npm run lint:md
  • npm run spell-check
  • npm run lint:frontmatter
  • npm run validate:skills
  • npm run lint:md-links
  • npm run lint:ps
  • npm run plugin:generate
  • npm run docs:test

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

This PR adds a decision-support agent, not a replacement for RPI or PR Review workflows. The value is independent multi-model reasoning with explicit synthesis.

…stion synthesis

- add council agent to synthesize responses from GPT-5.4, Opus 4.6, and Gemini 3.1 Pro
- implement individual councilor agents for each model
- update documentation to reflect new council structure and capabilities

🔍 - Generated by Copilot
@andrewDoing andrewDoing requested a review from a team as a code owner April 9, 2026 04:55
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.65%. Comparing base (a18ab1a) to head (93cfe2d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1327      +/-   ##
==========================================
+ Coverage   87.63%   87.65%   +0.01%     
==========================================
  Files          61       61              
  Lines        9328     9328              
==========================================
+ Hits         8175     8176       +1     
+ Misses       1153     1152       -1     
Flag Coverage Δ
pester 85.22% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new agent & subagents are a great idea.
What I'm interested in understanding better is how it fits into RPI, and which scenarios to use it for.

Documentation needed:

We also need to document this new option, otherwise it might be another agent folks don't fully grasp at what stage to use best. Optimally fully document a few key scenarios.

RPI integration points?

Can it be used after research/does a user select context files from the branch, where is the context typically for having the council subagents compare with?

Can we integrate it in Task Reviewer optionally? Is that the right fit as you'd have good context, but may lack the different implementation options.

Please also ensure you have ran /prompt-analyze prompt on the selected assets?
Finally, once the assets are added to the right collections and maturity, run the plugin regeneration.

kind: agent
- path: .github/agents/hve-core/doc-ops.agent.md
kind: agent
- path: .github/agents/hve-core/council.agent.md
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this new addition but I think we need to make sure we get some user feedback before pushing it into the main collection in stable.

Either, add maturity: experimental to each artifact, in all collections you touched.
Or add it to experimental.collection.yml instead?

kind: agent
- path: .github/agents/github/github-backlog-manager.agent.md
kind: agent
- path: .github/agents/hve-core/council.agent.md
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add maturity: experimental to each of the newly added items

@WilliamBerryiii WilliamBerryiii changed the title Add LLM Council agent and councilor subagents to hve-core feat(agents): add LLM Council agent and councilor subagents Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Add LLM Council agent for multi-model consensus synthesis

3 participants