[MDB-47576] Optimize GET_DATABASES_SQL

[UberDever]

Summary by Sourcery

Refine the GET_DATABASES_SQL query to handle replicated and non-replicated databases separately while preserving nullable engine_full handling.

[sourcery-ai]

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Refactors the GET_DATABASES_SQL query to explicitly split handling of Replicated and non-Replicated databases, ensuring correct nullable typing of engine_full and clearer filtering logic.

Flow diagram for updated GET_DATABASES_SQL filtering and engine_full assignment

flowchart TD
    A[system.databases] --> B[Filter name NOT IN excluded_system_dbs]
    B --> C[engine = Replicated]
    B --> D[engine != Replicated]

    C --> E("Select name, engine, metadata_path, uuid, CAST(engine_full, Nullable(String)) AS engine_full")
    D --> F("Select name, engine, metadata_path, uuid, CAST(NULL, Nullable(String)) AS engine_full")

    E --> G[UNION ALL]
    F --> G[UNION ALL]
    G --> H[FORMAT JSON result]

Loading

File-Level Changes

Change	Details	Files
Refine GET_DATABASES_SQL query to correctly handle Replicated vs non-Replicated engines while enforcing a nullable type for engine_full.	Replace conditional selection of engine_full with explicit CAST to Nullable(String). Restrict the first SELECT to only Replicated databases and preserve existing database name filters. Add a second SELECT for non-Replicated databases that sets engine_full to NULL cast as Nullable(String). Combine the two SELECTs with UNION ALL while retaining JSON output format.	ch_backup/clickhouse/control.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The name NOT IN (...) filter is duplicated across both branches of the UNION; consider factoring this into a common subquery or CTE to avoid repetition and keep the exclusion list maintainable.
• You can likely avoid the UNION by using a single SELECT with a conditional expression (e.g., if(engine = 'Replicated', CAST(engine_full, 'Nullable(String)'), CAST(NULL, 'Nullable(String)')) AS engine_full) which keeps the query simpler while preserving the desired type and behavior.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `name NOT IN (...)` filter is duplicated across both branches of the UNION; consider factoring this into a common subquery or CTE to avoid repetition and keep the exclusion list maintainable.
- You can likely avoid the UNION by using a single SELECT with a conditional expression (e.g., `if(engine = 'Replicated', CAST(engine_full, 'Nullable(String)'), CAST(NULL, 'Nullable(String)')) AS engine_full`) which keeps the query simpler while preserving the desired type and behavior.

## Individual Comments

### Comment 1
<location path="ch_backup/clickhouse/control.py" line_range="300-302" />
<code_context>
+        CAST(engine_full, 'Nullable(String)') AS engine_full
     FROM system.databases
-    WHERE name NOT IN ('system', '_temporary_and_external_tables', 'information_schema', 'INFORMATION_SCHEMA', '{system_db}')
+    WHERE 
+        engine = 'Replicated' AND
+        name NOT IN ('system', '_temporary_and_external_tables', 'information_schema', 'INFORMATION_SCHEMA', '{system_db}')
+    UNION ALL
+    SELECT
</code_context>
<issue_to_address>
**suggestion (performance):** The duplicated filters across both SELECTs may cause unnecessary double scanning of system.databases.

Both UNION ALL branches reapply the same `name NOT IN (...)` filter and differ only by the engine condition and `engine_full`. As a result, `system.databases` is scanned twice with nearly identical predicates. If this query runs on large deployments, consider consolidating into a single SELECT (e.g., with a conditional `engine_full` expression) or introducing a subquery so these filters are evaluated only once while preserving the Nullable semantics.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}