Add Discord channel monitoring/logging feature for bot operations

[timothyfroehlich]

Feature Request

Add the ability to monitor bot operations directly in a Discord channel, providing real-time visibility into bot health, startup/shutdown events, and errors.

Proposed Commands

!monitor_channel [enable|disable]

Configure the current channel to receive bot operational messages.

Usage:

!monitor_channel enable   # Enable monitoring in current channel
!monitor_channel disable  # Disable monitoring in current channel
!monitor_channel          # Show current monitoring status

!monitor_verbose [on|off]

Toggle detailed loop status messages during normal operations.

Usage:

!monitor_verbose on   # Enable verbose monitoring (loop iterations, error counts)
!monitor_verbose off  # Disable verbose monitoring (only critical events)
!monitor_verbose      # Show current verbose status

Messages to Post

Critical Events (Always Posted to Monitor Channel)

• Bot Startup: "🟢 Bot started successfully at 2025-06-25 20:15:33 UTC"
• Bot Shutdown: "🔴 Bot shutting down gracefully at 2025-06-25 20:45:22 UTC"
• Critical Errors: "🚨 Critical error in monitor loop: [error details]"
• Monitor Loop Crashes: "💥 Monitor loop crashed and restarted: [error details]"
• Database Connection Issues: "⚠️ Database connection lost/restored"

Verbose Status Messages (Only When Enabled)

• Loop Iterations: "🔄 Monitor loop Bot fails to handle newlines in messages (export command failure) #42 completed: 3 channels polled, 0 errors, 2.3s duration"
• Error Count Updates: "⚠️ Total errors: 5 (2 new this hour)"
• Performance Alerts: "🐌 Loop iteration took 45s (>30s threshold)"
• Channel Status: "📊 Active monitoring: 8 channels, 24 targets"

Implementation Details

Database Schema

Add to ChannelConfig or create new table:

-- Option 1: Extend ChannelConfig
ALTER TABLE channel_configs ADD COLUMN is_monitor_channel BOOLEAN DEFAULT FALSE;
ALTER TABLE channel_configs ADD COLUMN monitor_verbose BOOLEAN DEFAULT FALSE;

-- Option 2: Separate table for operational monitoring
CREATE TABLE monitor_channels (
    channel_id BIGINT PRIMARY KEY,
    guild_id BIGINT NOT NULL,
    is_enabled BOOLEAN DEFAULT TRUE,
    verbose_mode BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMP DEFAULT NOW()
);

Code Integration Points

1. Startup/Shutdown: Hook into main.py startup and graceful shutdown
2. Monitor Loop: Add monitoring messages to existing monitor loop
3. Error Handling: Integrate with existing error logging in monitor cog
4. Rate Limiting: Prevent spam by throttling verbose messages

Configuration Options

• Rate Limiting: Max 1 verbose message per minute per channel
• Error Thresholds: Only post error updates when count increases significantly
• Message Formatting: Clean, consistent formatting with emojis for visibility

Example Usage Workflow

1. Admin runs !monitor_channel enable in #bot-monitoring channel
2. Bot responds: "✅ Monitoring enabled for #bot-monitoring"
3. Bot posts startup message when restarted
4. Admin runs !monitor_verbose on for detailed loop status
5. Bot posts loop iteration summaries every few minutes
6. When errors occur, bot immediately posts to monitoring channel

Benefits

• Real-time visibility into bot health and operations
• Immediate alerts for critical issues requiring attention
• Deployment verification through startup/shutdown messages
• Performance monitoring via loop timing and error tracking
• Debugging assistance with verbose operational details

Security Considerations

• Permissions: Only users with manage server permissions can configure monitoring
• Channel Validation: Verify bot has send message permissions in monitor channel
• Error Sanitization: Don't expose sensitive information in error messages
• Rate Limiting: Prevent spam/abuse of monitoring features

Priority

Medium-High - Significantly improves operational visibility and debugging capabilities.

Related Issues

• Monitor loop waits 60 minutes for first check after bot restart #49 (Monitor loop startup delay) - Monitoring would help verify when first checks run
• Add bot runtime display to !status command #48 (Bot runtime in status) - Complementary operational visibility features