Skip to content

Add Comment Moderation experiment #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Jameswlepage wants to merge 6 commits into
base: develop
Choose a base branch
from
Draft

Add Comment Moderation experiment #155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
0a7405d
Add Comment Moderation experiment
Jameswlepage Dec 19, 2025
a27bb86
Merge branch 'develop' into feature/comment-moderation
jeffpaul Dec 19, 2025
03966de
Update readme with comment moderation feature
jeffpaul Dec 20, 2025
0ee4cd8
Create screenshot-#.png
jeffpaul Dec 20, 2025
9eae9fd
Merge branch 'develop' into feature/comment-moderation
dkotter Feb 3, 2026
5443ec1
Ensure we use the proper function to avoid errors
dkotter Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .wordpress-org/screenshot-#.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
34 changes: 34 additions & 0 deletions docs/experiments/comment-moderation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Comment Moderation

## Summary
Adds AI-powered sentiment analysis, toxicity scoring, and reply suggestions to the classic Comments screen. Moderators can see badges directly in `edit-comments.php`, run bulk analysis, and request suggested replies without leaving wp-admin.

## Key Hooks & Entry Points
- `WordPress\AI\Experiments\Comment_Moderation\Comment_Moderation::register()` wires everything once the experiment is enabled:
- `wp_abilities_api_init` → registers `ai/comment-analysis` and `ai/reply-suggestion` abilities (`includes/Abilities/Comment_Moderation/*.php`).
- `manage_edit-comments_columns`, `manage_comments_custom_column` → inject sentiment/toxicity columns.
- `bulk_actions-edit-comments`, `handle_bulk_actions-edit-comments`, `admin_notices` → add the “Analyze with AI” bulk flow and status notices.
- `comment_row_actions` → adds the “AI Reply” row action.
- `admin_enqueue_scripts` → enqueues the React bundle on `edit-comments.php`.
- `admin_head-edit-comments.php` → prints inline badge styles so they render even when JS fails.
- REST and comment-meta updates happen via the two abilities; the experiment itself only orchestrates UI + enqueue points.

## Assets & Data Flow
1. `enqueue_assets()` loads `experiments/comment-moderation` (`src/experiments/comment-moderation/index.tsx`) and localizes `window.CommentModerationData` with `enabled` + nonce.
2. The React entry mounts two controllers:
- `LazyAnalysisController` polls for comments that need analysis and calls `runAbility( 'ai/comment-analysis' )`, updating comment meta and refreshing rows in place.
- `ReplyModalController` opens a modal when an “AI Reply” row action is clicked, calling `runAbility( 'ai/reply-suggestion' )` to fetch draft replies the moderator can paste.
3. Both controllers rely on the shared `run-ability.ts` helper so they can use the Abilities API client when available or fall back to REST calls.
4. Ability responses are persisted via comment meta (`_ai_toxicity_score`, `_ai_sentiment`, `_ai_analysis_status`, `_ai_analyzed_at`), which the PHP column renderers read to display badges.

## Testing
1. Enable Experiments globally and toggle **Comment Moderation** under `Settings → AI Experiments`.
2. Visit `Comments → All Comments`. Pending comments should show “Analyze with AI” badges; clicking one should enqueue an analysis request and update the badge once complete.
3. Select multiple comments, choose the “Analyze with AI” bulk action, and confirm the inline notice reports how many were queued.
4. Approve a comment and click its “AI Reply” row action. The modal should display suggested replies; applying one should copy it into the WordPress reply form.
5. Toggle the experiment off and reload the page—columns, badges, row/bulk actions, and scripts should disappear.

## Notes
- The experiment only runs for users with `moderate_comments`.
- Analysis locks each comment while it is processing to prevent duplicate requests.
- Replies and analysis rely on AI credentials; without valid credentials the whole experiment remains disabled via the shared experiment toggle logic.
226 changes: 226 additions & 0 deletions includes/Abilities/Comment_Moderation/Comment_Analysis.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
<?php
/**
* Comment Analysis WordPress Ability implementation.
*
* @package WordPress\AI
*/

declare( strict_types=1 );

namespace WordPress\AI\Abilities\Comment_Moderation;

use WP_Error;
use WordPress\AI\Abstracts\Abstract_Ability;
use WordPress\AI\Experiments\Comment_Moderation\Comment_Moderation;
use WordPress\AI_Client\AI_Client;

/**
* Comment Analysis WordPress Ability.
*
* Analyzes comments for toxicity and sentiment using AI.
*
* @since 0.1.0
*/
class Comment_Analysis extends Abstract_Ability {

/**
* Returns the input schema of the ability.
*
* @since 0.1.0
*
* @return array<string, mixed> The input schema of the ability.
*/
protected function input_schema(): array {
return array(
'type' => 'object',
'properties' => array(
'comment_id' => array(
'type' => 'integer',
'sanitize_callback' => 'absint',
'description' => esc_html__( 'The ID of the comment to analyze.', 'ai' ),
'required' => true,
),
),
'required' => array( 'comment_id' ),
);
}

/**
* Returns the output schema of the ability.
*
* @since 0.1.0
*
* @return array<string, mixed> The output schema of the ability.
*/
protected function output_schema(): array {
return array(
'type' => 'object',
'properties' => array(
'comment_id' => array(
'type' => 'integer',
'description' => esc_html__( 'The analyzed comment ID.', 'ai' ),
),
'toxicity_score' => array(
'type' => 'number',
'minimum' => 0,
'maximum' => 1,
'description' => esc_html__( 'Toxicity score from 0 (not toxic) to 1 (highly toxic).', 'ai' ),
),
'sentiment' => array(
'type' => 'string',
'enum' => array( 'positive', 'negative', 'neutral' ),
'description' => esc_html__( 'The sentiment of the comment.', 'ai' ),
),
),
);
}

/**
* Executes the ability with the given input arguments.
*
* @since 0.1.0
*
* @param mixed $input The input arguments to the ability.
* @return array{comment_id: int, toxicity_score: float, sentiment: string}|\WP_Error The result of the ability execution.
*/
protected function execute_callback( $input ) {
$comment_id = absint( $input['comment_id'] ?? 0 );

if ( ! $comment_id ) {
return new WP_Error(
'missing_comment_id',
esc_html__( 'Comment ID is required.', 'ai' )
);
}

$comment = get_comment( $comment_id );

if ( ! $comment ) {
return new WP_Error(
'comment_not_found',
sprintf(
/* translators: %d: Comment ID. */
esc_html__( 'Comment with ID %d not found.', 'ai' ),
$comment_id
)
);
}

// Check if already being processed (lock mechanism).
$current_status = get_comment_meta( $comment_id, Comment_Moderation::META_ANALYSIS_STATUS, true );

if ( Comment_Moderation::STATUS_PROCESSING === $current_status ) {
return new WP_Error(
'already_processing',
esc_html__( 'This comment is already being analyzed.', 'ai' )
);
}

// Set status to processing.
update_comment_meta( $comment_id, Comment_Moderation::META_ANALYSIS_STATUS, Comment_Moderation::STATUS_PROCESSING );

// Analyze the comment.
$result = $this->analyze_comment( $comment->comment_content, $comment->comment_author );

if ( is_wp_error( $result ) ) {
// Mark as failed.
update_comment_meta( $comment_id, Comment_Moderation::META_ANALYSIS_STATUS, Comment_Moderation::STATUS_FAILED );
return $result;
}

// Store the results.
update_comment_meta( $comment_id, Comment_Moderation::META_TOXICITY_SCORE, $result['toxicity_score'] );
update_comment_meta( $comment_id, Comment_Moderation::META_SENTIMENT, $result['sentiment'] );
update_comment_meta( $comment_id, Comment_Moderation::META_ANALYSIS_STATUS, Comment_Moderation::STATUS_COMPLETE );
update_comment_meta( $comment_id, Comment_Moderation::META_ANALYZED_AT, time() );

return array(
'comment_id' => $comment_id,
'toxicity_score' => $result['toxicity_score'],
'sentiment' => $result['sentiment'],
);
}

/**
* Returns the permission callback of the ability.
*
* @since 0.1.0
*
* @param mixed $input The input arguments to the ability.
* @return bool|\WP_Error True if the user has permission, WP_Error otherwise.
*/
protected function permission_callback( $input ) {
if ( ! current_user_can( 'moderate_comments' ) ) {
return new WP_Error(
'insufficient_capabilities',
esc_html__( 'You do not have permission to analyze comments.', 'ai' )
);
}

return true;
}

/**
* Returns the meta of the ability.
*
* @since 0.1.0
*
* @return array<string, mixed> The meta of the ability.
*/
protected function meta(): array {
return array(
'show_in_rest' => true,
);
}

/**
* Analyzes a comment for toxicity and sentiment.
*
* @since 0.1.0
*
* @param string $content The comment content.
* @param string $author The comment author name.
* @return array{toxicity_score: float, sentiment: string}|\WP_Error The analysis result.
*/
private function analyze_comment( string $content, string $author ) {
$prompt = sprintf(
"Comment by %s:\n\"\"\"%s\"\"\"",
$author,
$content
);

$result = AI_Client::prompt_with_wp_error( $prompt )
->using_system_instruction( $this->get_system_instruction() )
->using_model_preference( ...$this->get_model_preferences() )
->generate_text();

if ( is_wp_error( $result ) ) {
return $result;
}

// Parse the JSON response.
$parsed = json_decode( $result, true );

if ( json_last_error() !== JSON_ERROR_NONE || ! is_array( $parsed ) ) {
return new WP_Error(
'parse_error',
esc_html__( 'Failed to parse AI response.', 'ai' )
);
}

// Validate and sanitize the response.
$toxicity_score = isset( $parsed['toxicity_score'] )
? max( 0, min( 1, (float) $parsed['toxicity_score'] ) )
: 0;

$valid_sentiments = array( 'positive', 'negative', 'neutral' );
$sentiment = isset( $parsed['sentiment'] ) && in_array( $parsed['sentiment'], $valid_sentiments, true )
? $parsed['sentiment']
: 'neutral';

return array(
'toxicity_score' => $toxicity_score,
'sentiment' => $sentiment,
);
}
}
Loading
Loading