Add @proxysql_min_version tag to groups.json for version-aware test execution

[rahim-kanji]

Problem

ProxySQL now ships multiple major versions (v3.0.x, v3.1.x, v4.0.x) with different feature sets. The TAP test suite has tests for features that only exist in newer versions (e.g., FFTO in 3.1+, GenAI/MCP in 4.0+), but there is no mechanism to skip them when running CI against an older version. This causes false-positive failures.

Proposed Solution

Add a @proxysql_min_version:X.Y metadata tag in groups.json entries. Tests without the tag default to running on all versions.

Example:

"test_ffto_mysql-t" : [ "legacy-g4", "mysql84-g4", "@proxysql_min_version:3.1" ]
"mcp_module-t" : [ "ai-g1", "@proxysql_min_version:4.0" ]
"test_binlog_something-t" : [ "legacy-g1", "mysql84-g1" ]

Implementation

1. groups.json — add @proxysql_min_version:X.Y tag to version-specific test entries
2. proxysql-tester.py — at startup, detect ProxySQL binary version; when loading groups.json, parse @proxysql_min_version tags and skip tests where binary version < min_version (log as "SKIP (requires vX.Y+)")
3. check_groups.py — teach it to ignore @-prefixed entries when validating group names

No changes needed to group directories, env.sh, infras.lst, hooks, CI workflows, or run-tests-isolated.bash.

Tests Requiring Tags

@proxysql_min_version:3.1 — FFTO/TSDB (17 tests)

Test	Current Groups
test_ffto_bypass-t	legacy-g4, mysql84-g4
test_ffto_mysql-t	legacy-g4, mysql84-g4
test_ffto_mysql_bypass_recovery-t	legacy-g4, mysql84-g4
test_ffto_mysql_concurrent-t	legacy-g4, mysql84-g4
test_ffto_mysql_large_queries-t	legacy-g4, mysql84-g4
test_ffto_mysql_large_resultsets-t	legacy-g4, mysql84-g4
test_ffto_mysql_mixed_protocol-t	legacy-g4, mysql84-g4
test_ffto_mysql_transactions-t	legacy-g4, mysql84-g4
test_ffto_pgsql-t	legacy-g4, mysql84-g4
test_ffto_pgsql_command_types-t	legacy-g4, mysql84-g4
test_ffto_pgsql_concurrent-t	legacy-g4, mysql84-g4
test_ffto_pgsql_errors-t	legacy-g4, mysql84-g4
test_ffto_pgsql_large_resultsets-t	legacy-g4, mysql84-g4
test_ffto_pgsql_mixed_protocol-t	legacy-g4, mysql84-g4
test_ffto_pgsql_pipeline-t	legacy-g4, mysql84-g4
test_ffto_pgsql_stmt_portal-t	legacy-g4, mysql84-g4
ffto_protocol_unit-t	unit-tests-g1

@proxysql_min_version:4.0 — GenAI/MCP/AI (48 tests)

All currently in ai-g1 group:

Category	Tests
AI core (3)	ai_error_handling_edge_cases-t, ai_llm_retry_scenarios-t, ai_validation-t
Anomaly (3)	anomaly_detection-t, anomaly_detection_integration-t, anomaly_detector_unit-t
GenAI (14)	genai_anomaly_unit-t, genai_async-t, genai_discovery_schema_unit-t, genai_embedding_rerank-t, genai_fts_string_unit-t, genai_live_validation-t, genai_llm_clients_unit-t, genai_mcp_endpoint_unit-t, genai_mcp_thread_unit-t, genai_module-t, genai_mysql_catalog_unit-t, genai_query_handler_unit-t, genai_stats_parsing_unit-t, genai_thread_unit-t
LLM (1)	llm_bridge_accuracy-t
MCP (18)	mcp_module-t, mcp_mysql_concurrency_stress-t, mcp_pgsql_concurrency_stress-t, mcp_mixed_mysql_pgsql_concurrency_stress-t, mcp_mixed_stats_cap_churn-t, mcp_mixed_stats_profile_matrix-t, mcp_query_rules-t, mcp_query_run_sql_readonly-t, mcp_runtime_variables-t, mcp_semantic_lifecycle-t, mcp_show_connections_commands_inmemory-t, mcp_show_queries_topk-t, mcp_stats_refresh-t, test_mcp_claude_headless_flow-t, test_mcp_llm_discovery_phaseb-t, test_mcp_query_rules-t, test_mcp_rag_metrics-t, test_mcp_static_harvest-t
NL2SQL (5)	nl2sql_integration-t, nl2sql_internal-t, nl2sql_model_selection-t, nl2sql_prompt_builder-t, nl2sql_unit_base-t
TSDB (2)	test_tsdb_api-t, test_tsdb_variables-t
Vector (2)	vector_db_performance-t, vector_features-t

Version Mapping

ProxySQL Version	Build Flag	Features
v3.0.x	(default)	Core proxy — all untagged tests
v3.1.x	PROXYSQL31=1	+ FFTO, TSDB
v4.0.x	PROXYSQLGENAI=1	+ GenAI, MCP, Anomaly Detection, NL2SQL, Vector

Acceptance Criteria

• All 65 version-specific tests tagged in groups.json
• proxysql-tester.py detects binary version and skips tests with higher @proxysql_min_version
• Skipped tests logged clearly (not silent)
• check_groups.py ignores @-prefixed metadata entries
• CI passes on v3.0.x branch without false-positive failures from 3.1+/4.0+ tests
• No changes to group directories, hooks, or CI workflow files

[coderabbitai]

🔗 Related PRs

#5293 - groups.json: add missing test [merged]
#5404 - Miscellaneous TAP test fixes (2026-02-21) [merged]
#5484 - Fix TSDB TAP tests for Docker-based CI [merged]
#5506 - HostGroups Manager unit tests (Phase 2.6) [merged]
#5520 - Add SKIP_PROXYSQL for infrastructure-free test groups (unit tests) [merged]
#5538 - CI test fixes: cluster sync, plan counts, test group assignments, and diagnostics [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!