Skip to content

docs: Add suggestion quality calibration to pilot guide #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
promptless wants to merge 2 commits into
base: main
Choose a base branch
from
Open

docs: Add suggestion quality calibration to pilot guide #164

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ad5e68
Add suggestion quality calibration to pilot guide
promptless[bot] Dec 6, 2025
a3d01ca
Sync documentation updates
promptless[bot] Dec 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions fern/docs/pilot-guide.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,15 @@ Since everyone has different expectations for Promptless, suggestion quality is
During your workshop, we'll discuss what "quality" means for your team—technical accuracy, style guide compliance, or catching edge cases. Understanding your quality criteria helps us make Promptless better for you.
</Tip>

#### Understanding Quality Ratings

To help you calibrate what different quality levels mean for your team:

* **"9-10":** Most suggestions from Promptless are relevant, and typically can be accepted with little to no edits.
* **"7-8":** Most suggestions are accurate and relevant, requiring only moderate editing. You're refining style, adding specific details, or adjusting tone rather than rewriting entire sections. Promptless meaningfully reduces your documentation workload.
* **"4-6":** Suggestions occasionally catch relevant changes and can serve as a starting point, but require significant editing. You're rewriting content, fixing inaccuracies, or filling in missing context. Promptless saves some time, but doesn't dramatically change your workflow.
* **"0-3":** Promptless is creating negative value. Most/all suggestions are not good enough to be a useful starting point for doc updates. Even if it's occasionally bringing up things that are relevant, I'd rather start from scratch myself.

## Documentation Improvement Projects

Sometimes teams need foundational work before piloting—like documenting major missing features, completing a large refactor, or creating documentation from scratch.
Expand Down