From 75ff0de764280e658e4d6fd7bbb536daad1f724b Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Sun, 16 Aug 2026 05:12:39 +0000 Subject: [PATCH 1/5] docs: design AI GCOV TAP shards --- .../specs/2026-08-16-ai-gcov-shards-design.md | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md diff --git a/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md b/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md new file mode 100644 index 0000000000..4d569774c6 --- /dev/null +++ b/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md @@ -0,0 +1,96 @@ +# AI GCOV TAP Shards Design + +## Goal + +Run every registered AI TAP in GitHub Actions and upload its GCOV data without +turning one job into an excessively long serial run. Add a focused TAP that +executes `PROXYSQLTEST 52` and verifies that its temporary configuration is +cleaned up. + +## Design + +The existing `ai-g1` registration contains 44 tests but has no CI workflow. +Split the registrations into two disjoint groups of 22. The split keeps related +test families together where possible and puts the largest stress workloads on +opposite shards: + +| Group | Main responsibilities | +| --- | --- | +| `ai-g1` | GenAI foundations; mixed MySQL/PgSQL concurrency; MCP module, rule, runtime, Top-K, and cap-churn checks; NL2SQL integration/internal; TSDB API; vector features. | +| `ai-g2` | Remaining GenAI units; dedicated MySQL and PgSQL concurrency; MCP SQL, semantic, connection, and refresh checks; NL2SQL model/prompt/base; TSDB variables; vector database performance. | + +`ai-g1` retains these 22 tests: + +``` +ai_llm_retry_scenarios-t +ai_validation-t +genai_config_query_unit-t +genai_discovery_schema_unit-t +genai_fts_string_unit-t +genai_llm_clients_unit-t +genai_mcp_endpoint_unit-t +genai_mcp_thread_unit-t +genai_module-t +llm_bridge_accuracy-t +mcp_mixed_mysql_pgsql_concurrency_stress-t +mcp_mixed_stats_cap_churn-t +mcp_mixed_stats_profile_matrix-t +mcp_module-t +mcp_query_rules-t +mcp_query_run_sql_readonly_bypass-t +mcp_runtime_variables-t +mcp_show_queries_topk-t +nl2sql_integration-t +nl2sql_internal-t +test_tsdb_api-t +vector_features-t +``` + +All remaining current `ai-g1` registrations move to `ai-g2`. This preserves +exactly 22 tests in each group and ensures every test belongs to exactly one AI +shard. + +Each group needs an environment file. `ai-g1/env.sh` currently adds the +GenAI-plugin variables beyond the parent `ai/env.sh`; `ai-g2/env.sh` must set +the same variables so ProxySQL loads the plugin before the TAPs run. + +Add `CI-ai-g1.yml` and `CI-ai-g2.yml` on v3.0. Each is a standard +`workflow_run` caller, modeled on `CI-legacy-g2-genai.yml`, and calls a +same-named reusable workflow from `GH-Actions`. Add the corresponding reusable +workflows on `GH-Actions`, modeled on `ci-legacy-g2-genai.yml`, with these +differences only: + +- `TAP_GROUP` and `INFRA_ID` are the selected AI shard; +- workflow/check/artifact names identify the selected AI shard; +- the Codecov upload name is unique (`tap-ai-g1-coverage` or + `tap-ai-g2-coverage`). + +Both reusable workflows consume the existing `ubuntu24-tap-genai-gcov` build +handoff and start the inherited `ai` infrastructure: MySQL 8.4 and PostgreSQL +16. They run independently, so their infrastructure and coverage counters are +isolated. + +Once both callers exist, remove `ai` from +`ALLOWLIST_NO_WORKFLOW` in `test/tap/groups/lint_group_coverage.py`. + +For `PROXYSQLTEST 52`, create a dedicated MySQL TAP instead of adding a bare +command to `admin_various_commands2-t.cpp`. It records the starting number of +`mysql_servers` rows for hostgroup 5211, runs `PROXYSQLTEST 52`, requires an OK +response, and verifies that both the admin table and runtime view return to the +starting state. Register it in one existing GCOV-enabled MySQL 8.4 group. + +## Validation + +- JSON and group-registration linting pass. +- A small structural test verifies exactly 22 disjoint registrations in each + AI group and that both workflow callers name the matching reusable workflow. +- The focused `PROXYSQLTEST 52` TAP passes in its MySQL 8.4 group. +- Both AI workflows complete, upload distinct LCOV reports, and the Codecov + report shows nonzero coverage for the existing Top-K validator invoked by + `mcp_show_queries_topk-t`. + +## Non-goals + +The unreachable hostgroup benchmark code following its unconditional +`return 0;` statements is not made artificially reachable. It remains a +separate source-cleanup or explicit-exclusion decision. From 18ac34641310d6621e3b52f9639ea3804cb906e6 Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Sun, 16 Aug 2026 05:23:02 +0000 Subject: [PATCH 2/5] ci: split AI TAP coverage into two shards --- .github/workflows/CI-ai-g1.yml | 21 +++++ .github/workflows/CI-ai-g2.yml | 21 +++++ .github/workflows/CI-lint-groups-json.yml | 2 + test/tap/groups/ai-g2/env.sh | 20 +++++ test/tap/groups/groups.json | 44 ++++----- test/tap/groups/lint_group_coverage.py | 1 - test/tap/groups/test_ai_group_shards.py | 103 ++++++++++++++++++++++ 7 files changed, 189 insertions(+), 23 deletions(-) create mode 100644 .github/workflows/CI-ai-g1.yml create mode 100644 .github/workflows/CI-ai-g2.yml create mode 100644 test/tap/groups/ai-g2/env.sh create mode 100644 test/tap/groups/test_ai_group_shards.py diff --git a/.github/workflows/CI-ai-g1.yml b/.github/workflows/CI-ai-g1.yml new file mode 100644 index 0000000000..d9218f670b --- /dev/null +++ b/.github/workflows/CI-ai-g1.yml @@ -0,0 +1,21 @@ +name: CI-ai-g1 +run-name: '${{ github.event.workflow_run && github.event.workflow_run.head_branch || github.ref_name }} ${{ github.workflow }} ${{ github.event.workflow_run && github.event.workflow_run.head_sha || github.sha }}' + +on: + workflow_dispatch: + workflow_run: + workflows: [ CI-trigger ] + types: [ completed ] + +concurrency: + group: ${{ github.workflow }}-${{ github.event.workflow_run && github.event.workflow_run.head_branch || github.ref_name }} + cancel-in-progress: true + +jobs: + run: + if: ${{ github.event.workflow_run && github.event.workflow_run.conclusion == 'success' || ! github.event.workflow_run }} + permissions: write-all + uses: sysown/proxysql/.github/workflows/ci-ai-g1.yml@GH-Actions + secrets: inherit + with: + trigger: ${{ toJson(github) }} diff --git a/.github/workflows/CI-ai-g2.yml b/.github/workflows/CI-ai-g2.yml new file mode 100644 index 0000000000..36c90f478e --- /dev/null +++ b/.github/workflows/CI-ai-g2.yml @@ -0,0 +1,21 @@ +name: CI-ai-g2 +run-name: '${{ github.event.workflow_run && github.event.workflow_run.head_branch || github.ref_name }} ${{ github.workflow }} ${{ github.event.workflow_run && github.event.workflow_run.head_sha || github.sha }}' + +on: + workflow_dispatch: + workflow_run: + workflows: [ CI-trigger ] + types: [ completed ] + +concurrency: + group: ${{ github.workflow }}-${{ github.event.workflow_run && github.event.workflow_run.head_branch || github.ref_name }} + cancel-in-progress: true + +jobs: + run: + if: ${{ github.event.workflow_run && github.event.workflow_run.conclusion == 'success' || ! github.event.workflow_run }} + permissions: write-all + uses: sysown/proxysql/.github/workflows/ci-ai-g2.yml@GH-Actions + secrets: inherit + with: + trigger: ${{ toJson(github) }} diff --git a/.github/workflows/CI-lint-groups-json.yml b/.github/workflows/CI-lint-groups-json.yml index a155704894..f1fbd81632 100644 --- a/.github/workflows/CI-lint-groups-json.yml +++ b/.github/workflows/CI-lint-groups-json.yml @@ -23,6 +23,8 @@ jobs: || echo "::warning::could not fetch GH-Actions; the workflow-coverage check will skip" - name: Lint groups.json format run: python3 test/tap/groups/lint_groups_json.py + - name: Check AI TAP shard split + run: python3 test/tap/groups/test_ai_group_shards.py - name: Check every TAP source is registered in groups.json run: python3 test/tap/groups/check_groups.py --source - name: Check cluster simulator coverage contract diff --git a/test/tap/groups/ai-g2/env.sh b/test/tap/groups/ai-g2/env.sh new file mode 100644 index 0000000000..c48a0fc63e --- /dev/null +++ b/test/tap/groups/ai-g2/env.sh @@ -0,0 +1,20 @@ +# AI-g2 Subgroup Environment Configuration +# Inherits from parent ai group - duplicated here for POSIX sh compatibility + +export DEFAULT_MYSQL_INFRA="infra-mysql84" +export DEFAULT_PGSQL_INFRA="docker-pgsql16-single" + +export TAP_MCPPORT="${TAP_MCPPORT:-6071}" +export MCP_TARGET_ID="${MCP_TARGET_ID:-tap_mysql_default}" +export MCP_AUTH_PROFILE_ID="${MCP_AUTH_PROFILE_ID:-tap_mysql_auth}" +export MCP_PGSQL_TARGET_ID="${MCP_PGSQL_TARGET_ID:-tap_pgsql_default}" +export MCP_PGSQL_AUTH_PROFILE_ID="${MCP_PGSQL_AUTH_PROFILE_ID:-tap_pgsql_auth}" +export MCP_MYSQL_HOSTGROUP_ID="${MCP_MYSQL_HOSTGROUP_ID:-9100}" +export MCP_PGSQL_HOSTGROUP_ID="${MCP_PGSQL_HOSTGROUP_ID:-9200}" + +export MYSQL_DATABASE="${MYSQL_DATABASE:-test}" +export PGSQL_DATABASE="${PGSQL_DATABASE:-postgres}" + +# The GenAI/MCP/RAG/LLM features are supplied by plugins/genai/. +export PROXYSQL_LOAD_GENAI_PLUGIN=1 +export PROXYSQL_CONFIG_OVERRIDE="${WORKSPACE}/test/tap/groups/ai/proxysql-ci.cnf" diff --git a/test/tap/groups/groups.json b/test/tap/groups/groups.json index 361903314e..f9c8b17b79 100644 --- a/test/tap/groups/groups.json +++ b/test/tap/groups/groups.json @@ -8,7 +8,7 @@ "admin_various_commands-t" : [ "legacy-g1","mariadb10-galera-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql84-gr-g1","mysql90-g1","mysql90-gr-g1","mysql93-g1","mysql93-gr-g1","mysql95-g1","mysql95-gr-g1" ], "admin_various_commands2-t" : [ "legacy-g1","mariadb10-galera-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql84-gr-g1","mysql90-g1","mysql90-gr-g1","mysql93-g1","mysql93-gr-g1","mysql95-g1","mysql95-gr-g1" ], "admin_various_commands3-t" : [ "legacy-g1","mariadb10-galera-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql84-gr-g1","mysql90-g1","mysql90-gr-g1","mysql93-g1","mysql93-gr-g1","mysql95-g1","mysql95-gr-g1" ], - "ai_error_handling_edge_cases-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "ai_error_handling_edge_cases-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "ai_llm_retry_scenarios-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "ai_validation-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "auth_unit-t" : [ "unit-tests-g1" ], @@ -46,14 +46,14 @@ "genai_mcp_endpoint_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "genai_mcp_thread_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "genai_module-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "genai_mysql_catalog_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "genai_mysql_catalog_unit-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "genai_plugin_anomaly_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], "genai_plugin_backend_client_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], "genai_plugin_load_unit-t" : [ "unit-tests-g1","@proxysql_min_version:4.0" ], - "genai_query_handler_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "genai_rag_fetch_from_source_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "genai_stats_parsing_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "genai_thread_unit-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "genai_query_handler_unit-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "genai_rag_fetch_from_source_unit-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "genai_stats_parsing_unit-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "genai_thread_unit-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "glovars_unit-t" : [ "unit-tests-g1" ], "gtid_server_data_unit-t" : [ "unit-tests-g1" ], "gtid_set_unit-t" : [ "unit-tests-g1" ], @@ -74,16 +74,16 @@ "mcp_mixed_stats_cap_churn-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "mcp_mixed_stats_profile_matrix-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "mcp_module-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_mysql_concurrency_stress-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_pgsql_concurrency_stress-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "mcp_mysql_concurrency_stress-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "mcp_pgsql_concurrency_stress-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "mcp_query_rules-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_query_run_sql_readonly-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "mcp_query_run_sql_readonly-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "mcp_query_run_sql_readonly_bypass-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "mcp_runtime_variables-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_semantic_lifecycle-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_show_connections_commands_inmemory-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "mcp_semantic_lifecycle-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "mcp_show_connections_commands_inmemory-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "mcp_show_queries_topk-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "mcp_stats_refresh-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "mcp_stats_refresh-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "monitor_health_unit-t" : [ "unit-tests-g1" ], "multiple_prepared_statements-t" : [ "legacy-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql90-g1","mysql95-g1" ], "mysql-fast_forward-t" : [ "legacy-g1","mysql-auto_increment_delay_multiplex=0-g1","mysql-multiplexing=false-g1","mysql-query_digests=0-g1","mysql-query_digests_keep_comment=1-g1","mysql84-g1","mysql90-g1","mysql95-g1" ], @@ -150,9 +150,9 @@ "mysqlx_tls_unit-t" : [ "mysqlx-tsan-g1","unit-tests-g1","@proxysql_min_version:4.0" ], "nl2sql_integration-t" : [ "ai-g1","@proxysql_min_version:4.0" ], "nl2sql_internal-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "nl2sql_model_selection-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "nl2sql_prompt_builder-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "nl2sql_unit_base-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "nl2sql_model_selection-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "nl2sql_prompt_builder-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "nl2sql_unit_base-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "ok_packet_mixed_queries-t" : [ "legacy-g10","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g4","mysql90-g4","mysql95-g4" ], "parsersql_digest_test-t" : [ "unit-tests-g1" ], "parsersql_unit-t" : [ "unit-tests-g1" ], @@ -430,10 +430,10 @@ "test_match_eof_conn_cap_libmariadb-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g4","mysql90-g4","mysql95-g4" ], "test_match_eof_conn_cap_libmysql-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g4","mysql90-g4","mysql95-g4" ], "test_max_transaction_time-t" : [ "legacy-g8","mysql-auto_increment_delay_multiplex=0-g3","mysql-multiplexing=false-g3","mysql-query_digests=0-g3","mysql-query_digests_keep_comment=1-g3","mysql84-g8","mysql90-g3","mysql95-g3" ], - "test_mcp_claude_headless_flow-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "test_mcp_llm_discovery_phaseb-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "test_mcp_rag_metrics-t" : [ "ai-g1","@proxysql_min_version:4.0" ], - "test_mcp_static_harvest-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "test_mcp_claude_headless_flow-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "test_mcp_llm_discovery_phaseb-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "test_mcp_rag_metrics-t" : [ "ai-g2","@proxysql_min_version:4.0" ], + "test_mcp_static_harvest-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "test_mysql_connect_retries-t" : [ "legacy-g8","mysql-auto_increment_delay_multiplex=0-g3","mysql-multiplexing=false-g3","mysql-query_digests=0-g3","mysql-query_digests_keep_comment=1-g3","mysql84-g8","mysql90-g3","mysql95-g3" ], "test_mysql_connect_retries_delay-t" : [ "legacy-g8","mysql-auto_increment_delay_multiplex=0-g3","mysql-multiplexing=false-g3","mysql-query_digests=0-g3","mysql-query_digests_keep_comment=1-g3","mysql84-g8","mysql90-g3","mysql95-g3" ], "test_mysql_hostgroup_attributes-1-t" : [ "legacy-g8","mysql-auto_increment_delay_multiplex=0-g3","mysql-multiplexing=false-g3","mysql-query_digests=0-g3","mysql-query_digests_keep_comment=1-g3","mysql84-g8","mysql90-g3","mysql95-g3" ], @@ -517,13 +517,13 @@ "test_ssl_fast_forward-3_libmysql-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_ssl_large_query-1-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_ssl_large_query-2-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], - "test_stats_mcp_tables-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "test_stats_mcp_tables-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "test_stats_proxysql_message_metrics-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_thread_conn_dist-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_throttle_max_bytes_per_second_to_client-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_tls_stats-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "test_tsdb_api-t" : [ "ai-g1","@proxysql_min_version:3.1" ], - "test_tsdb_variables-t" : [ "ai-g1","@proxysql_min_version:3.1" ], + "test_tsdb_variables-t" : [ "ai-g2","@proxysql_min_version:3.1" ], "test_unexpected_packet_log_attribution-t" : [ "legacy-g6" ], "test_unhealthy_connection_log_attribution-t" : [ "legacy-g6" ], "test_unshun_algorithm-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], @@ -533,6 +533,6 @@ "test_wexecvp_syscall_failures-t" : [ "legacy-g9","mysql-auto_increment_delay_multiplex=0-g4","mysql-multiplexing=false-g4","mysql-query_digests=0-g4","mysql-query_digests_keep_comment=1-g4","mysql84-g9","mysql90-g4","mysql95-g4" ], "transaction_state_unit-t" : [ "unit-tests-g1" ], "unit-strip_schema_from_query-t" : [ "unit-tests-g1" ], - "vector_db_performance-t" : [ "ai-g1","@proxysql_min_version:4.0" ], + "vector_db_performance-t" : [ "ai-g2","@proxysql_min_version:4.0" ], "vector_features-t" : [ "ai-g1","@proxysql_min_version:4.0" ] } diff --git a/test/tap/groups/lint_group_coverage.py b/test/tap/groups/lint_group_coverage.py index 401455a5eb..3da3a8299c 100755 --- a/test/tap/groups/lint_group_coverage.py +++ b/test/tap/groups/lint_group_coverage.py @@ -53,7 +53,6 @@ # this list as workflows are added (e.g. mysql90-gr/mysql93-gr/mysql95-gr # were removed from here once their CI-*.yml callers landed). ALLOWLIST_NO_WORKFLOW = { - "ai", "legacy-binlog", "mysql84-binlog", "mysql90", diff --git a/test/tap/groups/test_ai_group_shards.py b/test/tap/groups/test_ai_group_shards.py new file mode 100644 index 0000000000..71a112fdba --- /dev/null +++ b/test/tap/groups/test_ai_group_shards.py @@ -0,0 +1,103 @@ +#!/usr/bin/env python3 +"""Regression contract for the balanced, CI-wired AI TAP shards.""" + +import json +import unittest +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[3] +GROUPS_JSON = ROOT / "test/tap/groups/groups.json" + +EXPECTED_G1 = { + "ai_llm_retry_scenarios-t", + "ai_validation-t", + "genai_config_query_unit-t", + "genai_discovery_schema_unit-t", + "genai_fts_string_unit-t", + "genai_llm_clients_unit-t", + "genai_mcp_endpoint_unit-t", + "genai_mcp_thread_unit-t", + "genai_module-t", + "llm_bridge_accuracy-t", + "mcp_mixed_mysql_pgsql_concurrency_stress-t", + "mcp_mixed_stats_cap_churn-t", + "mcp_mixed_stats_profile_matrix-t", + "mcp_module-t", + "mcp_query_rules-t", + "mcp_query_run_sql_readonly_bypass-t", + "mcp_runtime_variables-t", + "mcp_show_queries_topk-t", + "nl2sql_integration-t", + "nl2sql_internal-t", + "test_tsdb_api-t", + "vector_features-t", +} + +EXPECTED_G2 = { + "ai_error_handling_edge_cases-t", + "genai_mysql_catalog_unit-t", + "genai_query_handler_unit-t", + "genai_rag_fetch_from_source_unit-t", + "genai_stats_parsing_unit-t", + "genai_thread_unit-t", + "mcp_mysql_concurrency_stress-t", + "mcp_pgsql_concurrency_stress-t", + "mcp_query_run_sql_readonly-t", + "mcp_semantic_lifecycle-t", + "mcp_show_connections_commands_inmemory-t", + "mcp_stats_refresh-t", + "nl2sql_model_selection-t", + "nl2sql_prompt_builder-t", + "nl2sql_unit_base-t", + "test_mcp_claude_headless_flow-t", + "test_mcp_llm_discovery_phaseb-t", + "test_mcp_rag_metrics-t", + "test_mcp_static_harvest-t", + "test_stats_mcp_tables-t", + "test_tsdb_variables-t", + "vector_db_performance-t", +} + + +class AiGroupShardTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + with GROUPS_JSON.open(encoding="utf-8") as groups_file: + cls.groups = json.load(groups_file) + + def members(self, group): + return {name for name, tags in self.groups.items() if group in tags} + + def test_groups_are_balanced_and_disjoint(self): + g1 = self.members("ai-g1") + g2 = self.members("ai-g2") + + self.assertSetEqual(g1, EXPECTED_G1) + self.assertSetEqual(g2, EXPECTED_G2) + self.assertEqual(len(g1), 22) + self.assertEqual(len(g2), 22) + self.assertSetEqual(g1 & g2, set()) + + def test_each_ai_tap_has_exactly_one_ai_shard(self): + for test_name in EXPECTED_G1 | EXPECTED_G2: + memberships = { + group + for group in ("ai-g1", "ai-g2") + if group in self.groups[test_name] + } + expected = {"ai-g1"} if test_name in EXPECTED_G1 else {"ai-g2"} + self.assertSetEqual(memberships, expected, test_name) + + def test_each_shard_has_a_matching_v3_caller(self): + for group in ("ai-g1", "ai-g2"): + caller = ROOT / ".github/workflows" / f"CI-{group}.yml" + self.assertTrue(caller.is_file(), caller) + self.assertIn( + f".github/workflows/ci-{group}.yml@GH-Actions", + caller.read_text(encoding="utf-8"), + ) + + +if __name__ == "__main__": + unittest.main() From c73c4aa7313dd92d266c7b194cf8abfb209f1e2a Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Sun, 16 Aug 2026 05:40:52 +0000 Subject: [PATCH 3/5] test: cover hostgroup balancing admin command --- test/tap/groups/groups.json | 1 + .../test_admin_hostgroup_balancing-t.cpp | 115 ++++++++++++++++++ 2 files changed, 116 insertions(+) create mode 100644 test/tap/tests/test_admin_hostgroup_balancing-t.cpp diff --git a/test/tap/groups/groups.json b/test/tap/groups/groups.json index f9c8b17b79..1c43291ec5 100644 --- a/test/tap/groups/groups.json +++ b/test/tap/groups/groups.json @@ -345,6 +345,7 @@ "statistics_unit-t" : [ "unit-tests-g1" ], "stmt_explain-t" : [ "legacy-g7","mysql-auto_increment_delay_multiplex=0-g2","mysql-multiplexing=false-g2","mysql-query_digests=0-g2","mysql-query_digests_keep_comment=1-g2","mysql84-g7","mysql90-g2","mysql95-g2" ], "test_PROXY_Protocol-t" : [ "legacy-g3","mysql-auto_increment_delay_multiplex=0-g3","mysql-multiplexing=false-g3","mysql-query_digests=0-g3","mysql-query_digests_keep_comment=1-g3","mysql84-g3","mysql90-g3","mysql95-g3" ], + "test_admin_hostgroup_balancing-t" : [ "mysql84-g1" ], "test_admin_stats-t" : [ "legacy-g7","mysql-auto_increment_delay_multiplex=0-g2","mysql-multiplexing=false-g2","mysql-query_digests=0-g2","mysql-query_digests_keep_comment=1-g2","mysql84-g7","mysql90-g2","mysql95-g2" ], "test_ansi_quotes_group_replication-t" : [ "mysql84-gr-g1","mysql90-gr-g1","mysql91-gr-g1","mysql92-gr-g1","mysql93-gr-g1","mysql95-gr-g1" ], "test_auth_methods-t" : [ "mysql-auto_increment_delay_multiplex=0-g2","mysql-multiplexing=false-g2","mysql-query_digests=0-g2","mysql-query_digests_keep_comment=1-g2","mysql84-g7","mysql90-g2","mysql95-g2" ], diff --git a/test/tap/tests/test_admin_hostgroup_balancing-t.cpp b/test/tap/tests/test_admin_hostgroup_balancing-t.cpp new file mode 100644 index 0000000000..8a05324c8e --- /dev/null +++ b/test/tap/tests/test_admin_hostgroup_balancing-t.cpp @@ -0,0 +1,115 @@ +/** + * @file test_admin_hostgroup_balancing-t.cpp + * @brief Validate the DEBUG hostgroup balancing builtin and its cleanup. + */ + +#include +#include +#include + +#include "mysql.h" + +#include "tap.h" +#include "command_line.h" + +static constexpr unsigned int kTestHostgroup = 5211; + +static bool read_hostgroup_count( + MYSQL* admin, const char* table, const char* condition, long long* count +) { + const std::string query = + "SELECT COUNT(*) FROM " + std::string(table) + + " WHERE hostgroup_id=" + std::to_string(kTestHostgroup) + + (condition != nullptr ? " AND " + std::string(condition) : ""); + if (mysql_query(admin, query.c_str()) != 0) { + diag("Failed to query %s: %s", table, mysql_error(admin)); + return false; + } + + MYSQL_RES* result = mysql_store_result(admin); + if (result == nullptr) { + diag("No result for %s: %s", table, mysql_error(admin)); + return false; + } + + bool success = false; + if (mysql_num_rows(result) == 1) { + MYSQL_ROW row = mysql_fetch_row(result); + if (row != nullptr && row[0] != nullptr) { + char* end = nullptr; + errno = 0; + const long long parsed = strtoll(row[0], &end, 10); + if (errno == 0 && end != row[0] && *end == '\0') { + *count = parsed; + success = true; + } + } + } + if (!success) { + diag("Unexpected COUNT(*) result from %s", table); + } + mysql_free_result(result); + return success; +} + +int main() { + plan(6); + + CommandLine cl; + if (cl.getEnv()) { + ok(false, "Load TAP environment"); + skip(5, "Cannot connect to admin without TAP environment"); + return exit_status(); + } + + MYSQL* admin = mysql_init(nullptr); + const bool connected = admin != nullptr && mysql_real_connect( + admin, cl.host, cl.admin_username, cl.admin_password, nullptr, cl.admin_port, nullptr, 0 + ) != nullptr; + ok(connected, "Connect to the ProxySQL admin interface"); + if (!connected) { + if (admin != nullptr) { + diag("Admin connection failed: %s", mysql_error(admin)); + mysql_close(admin); + } + skip(5, "Cannot exercise PROXYSQLTEST 52 without admin access"); + return exit_status(); + } + + long long before_admin = -1; + const bool before_admin_ok = read_hostgroup_count(admin, "mysql_servers", nullptr, &before_admin); + ok(before_admin_ok && before_admin == 0, + "Hostgroup %u is absent from mysql_servers before PROXYSQLTEST 52", kTestHostgroup); + + long long before_runtime = -1; + const bool before_runtime_ok = read_hostgroup_count( + admin, "runtime_mysql_servers", nullptr, &before_runtime + ); + ok(before_runtime_ok && before_runtime == 0, + "Hostgroup %u is absent from runtime_mysql_servers before PROXYSQLTEST 52", kTestHostgroup); + + const int command_rc = mysql_query(admin, "PROXYSQLTEST 52"); + ok(command_rc == 0, "PROXYSQLTEST 52 completes the hostgroup balancing validator"); + if (command_rc != 0) { + diag("PROXYSQLTEST 52 failed: %s", mysql_error(admin)); + } + + long long after_admin = -1; + const bool after_admin_ok = read_hostgroup_count(admin, "mysql_servers", nullptr, &after_admin); + ok(after_admin_ok && after_admin == 0, + "Hostgroup %u is removed from mysql_servers after PROXYSQLTEST 52", kTestHostgroup); + + long long after_runtime = -1; + long long offline_hard = -1; + const bool after_runtime_ok = read_hostgroup_count( + admin, "runtime_mysql_servers", nullptr, &after_runtime + ); + const bool offline_hard_ok = read_hostgroup_count( + admin, "runtime_mysql_servers", "status='OFFLINE_HARD'", &offline_hard + ); + ok(after_runtime_ok && offline_hard_ok && after_runtime == 3 && offline_hard == 3, + "PROXYSQLTEST 52 leaves exactly three simulated in-use servers OFFLINE_HARD in runtime"); + + mysql_close(admin); + return exit_status(); +} From a1752af1ddeec3c9d2271ad19254e4ca43bc4008 Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Sun, 16 Aug 2026 06:20:53 +0000 Subject: [PATCH 4/5] ci: harden AI GCOV shard callers --- .github/workflows/CI-ai-g1.yml | 8 ++++++-- .github/workflows/CI-ai-g2.yml | 8 ++++++-- .../specs/2026-08-16-ai-gcov-shards-design.md | 11 ++++++----- 3 files changed, 18 insertions(+), 9 deletions(-) diff --git a/.github/workflows/CI-ai-g1.yml b/.github/workflows/CI-ai-g1.yml index d9218f670b..10c8fc1f56 100644 --- a/.github/workflows/CI-ai-g1.yml +++ b/.github/workflows/CI-ai-g1.yml @@ -14,8 +14,12 @@ concurrency: jobs: run: if: ${{ github.event.workflow_run && github.event.workflow_run.conclusion == 'success' || ! github.event.workflow_run }} - permissions: write-all + permissions: + actions: read + checks: write + contents: read + id-token: write + packages: read uses: sysown/proxysql/.github/workflows/ci-ai-g1.yml@GH-Actions - secrets: inherit with: trigger: ${{ toJson(github) }} diff --git a/.github/workflows/CI-ai-g2.yml b/.github/workflows/CI-ai-g2.yml index 36c90f478e..8b2a6595ab 100644 --- a/.github/workflows/CI-ai-g2.yml +++ b/.github/workflows/CI-ai-g2.yml @@ -14,8 +14,12 @@ concurrency: jobs: run: if: ${{ github.event.workflow_run && github.event.workflow_run.conclusion == 'success' || ! github.event.workflow_run }} - permissions: write-all + permissions: + actions: read + checks: write + contents: read + id-token: write + packages: read uses: sysown/proxysql/.github/workflows/ci-ai-g2.yml@GH-Actions - secrets: inherit with: trigger: ${{ toJson(github) }} diff --git a/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md b/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md index 4d569774c6..d4d79eb970 100644 --- a/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md +++ b/docs/superpowers/specs/2026-08-16-ai-gcov-shards-design.md @@ -21,7 +21,7 @@ opposite shards: `ai-g1` retains these 22 tests: -``` +```text ai_llm_retry_scenarios-t ai_validation-t genai_config_query_unit-t @@ -74,10 +74,11 @@ Once both callers exist, remove `ai` from `ALLOWLIST_NO_WORKFLOW` in `test/tap/groups/lint_group_coverage.py`. For `PROXYSQLTEST 52`, create a dedicated MySQL TAP instead of adding a bare -command to `admin_various_commands2-t.cpp`. It records the starting number of -`mysql_servers` rows for hostgroup 5211, runs `PROXYSQLTEST 52`, requires an OK -response, and verifies that both the admin table and runtime view return to the -starting state. Register it in one existing GCOV-enabled MySQL 8.4 group. +command to `admin_various_commands2-t.cpp`. It requires hostgroup 5211 to be +absent before the command, runs `PROXYSQLTEST 52`, requires an OK response, and +verifies that `mysql_servers` is empty afterward while `runtime_mysql_servers` +contains exactly three `OFFLINE_HARD` rows. Register it in one existing +GCOV-enabled MySQL 8.4 group. ## Validation From 2a764bb3a089638170bdb2518e28eca184941737 Mon Sep 17 00:00:00 2001 From: Rene Cannao Date: Sun, 16 Aug 2026 06:23:51 +0000 Subject: [PATCH 5/5] ci: retrigger AI coverage shard checks