refactor(drive): unify AVG no-prove dispatch into a single count+sum walk

book/src/drive/average-index-examples.md

@@ -1702,3 +1702,9 @@ The split closely parallels the count and sum chapters — point lookups for Q1
 The chapter is grounded in the [`document_average_worst_case`](https://github.com/dashpay/platform/blob/v3.1-dev/packages/rs-drive/benches/document_average_worst_case.rs) bench's measured numbers — Q1–Q7 verify cleanly end-to-end against the shared root hash `8b15f732…ffc7`.
 
 A natural expansion follow-up (out of scope here): a worked example of "exact-precision" averages — for callers that need fractional averages (e.g. `avg = 50.7142857…` rather than `50.99`), the protocol-level approach is to return `(count, sum)` and let the client compute in its preferred numeric format (the chapter notes this in [Numerical Considerations](#numerical-considerations) above; a future expansion could walk through the fixed-point vs. floating-point trade-offs).
+
+### No-proof path: single-walk joint dispatch
+
+Both the prove path (above) and the no-proof path now read `(count, sum)` from each visited count-sum-bearing element in a **single** grovedb walk. The no-proof path used to compose parallel count + sum sub-requests and zip the responses; that worked under a shared read transaction (atomic) but did twice the grovedb work strictly necessary. As of [#3687](https://github.com/dashpay/platform/issues/3687) the joint dispatcher at `crate::query::drive_document_count_and_sum_query` walks PCPS / CountSumTree elements once per AVG no-prove query and folds `(count, sum)` in Rust — halving the per-query grovedb work.
+
+One forward-looking optimization remains open in grovedb: a no-prove engine-side `query_aggregate_count_and_sum` accumulator (the proof-side analog `AggregateCountAndSumOnRange` already exists and is what Q5 / Q7 above use). Today's no-prove flat-summed branch walks all matched PCPS elements and folds in Rust; an engine-side accumulator could collapse that fold the way `query_aggregate_sum` collapses the sum-side equivalent. Not a blocker — the Rust-side fold is correctness-equivalent and the per-element count-sum decode is cheap — but it would close the residual perf gap between the no-prove and prove paths.

packages/rs-drive/src/query/drive_document_average_query/mod.rs

@@ -15,15 +15,16 @@
 //! [`book/src/drive/average-index-examples.md`](../../../../../book/src/drive/average-index-examples.md)
 //! for the design and the grades-contract worked example.
 //!
-//! Wired end-to-end: the dispatcher composes the count + sum
-//! executors on the no-proof path under a shared read-transaction
-//! (see [`drive_dispatcher`] module docstring for the atomicity
-//! contract), and the prove path dispatches directly to the PCPS /
-//! primary-key proof executors. A planned follow-up tracked at
-//! [dashpay/platform#3687](https://github.com/dashpay/platform/issues/3687)
-//! will collapse the no-proof path's two-request composition into a
-//! single unified executor that reads both metrics from each visited
-//! PCPS element in one walk.
+//! Wired end-to-end: the dispatcher routes prove-true requests to the
+//! PCPS / primary-key proof executors, and prove-false requests to the
+//! joint single-walk count-and-sum dispatcher at
+//! [`crate::query::drive_document_count_and_sum_query`]. Both paths
+//! read `(count, sum)` from each visited count-sum-bearing element in
+//! a single grovedb walk — see the
+//! [`drive_dispatcher`](drive_dispatcher) module docstring for the
+//! routing details and
+//! [`crate::query::drive_document_count_and_sum_query`] for the
+//! no-prove perf / atomicity contract.
 
 #[cfg(feature = "server")]
 pub mod drive_dispatcher;

packages/rs-drive/src/query/drive_document_count_and_sum_query/drive_dispatcher.rs

@@ -0,0 +1,248 @@
+//! Joint count-and-sum dispatcher entry point for the AVG no-prove
+//! path.
+//!
+//! Resolves [issue #3687](https://github.com/dashpay/platform/issues/3687):
+//! consumes a [`DocumentAverageRequest`] with `prove = false` and
+//! routes it through one of three joint per-mode executors
+//! (`Total` / `PerInValue` / `RangeNoProof`), each of which walks
+//! grovedb once and reads `(count, sum)` from every visited
+//! count-sum-bearing element. The dispatcher delegates routing to
+//! sum's versioned mode-detection table — see [the routing
+//! table](crate::query::drive_document_sum_query::mode_detection) for
+//! the contract.
+//!
+//! # Perf characteristic
+//!
+//! One grovedb walk per AVG no-prove query (compared to the pre-#3687
+//! composition of two parallel walks zipped at the dispatcher). The
+//! `PerInValue` and compound-aggregate range branches still issue
+//! multiple per-In sub-reads, but each of those is a single grovedb
+//! call yielding `(count, sum)` together rather than the two parallel
+//! calls the composition did before.
+//!
+//! # Atomicity
+//!
+//! - `Total` and the distinct `RangeNoProof` branch issue exactly one
+//!   grovedb read each; atomicity is inherent (one read sees one
+//!   snapshot).
+//! - The `PerInValue` branch and (under the unified walk) the
+//!   compound-aggregate `RangeNoProof` branch issue multiple per-In
+//!   reads. Their executors open a short-lived shared read transaction
+//!   internally when `transaction.is_none()` so the per-In reads see a
+//!   consistent snapshot. The previous AVG dispatcher's shared-tx
+//!   plumbing moved into those executors.
+
+use super::super::drive_document_average_query::{
+    AverageMode, DocumentAverageRequest, DocumentAverageResponse,
+};
+use super::super::drive_document_sum_query::mode_detection::detect_sum_mode_from_inputs;
+use super::super::drive_document_sum_query::{DocumentSumMode, RangeSumOptions, SumMode};
+use crate::drive::Drive;
+use crate::error::Error;
+use dpp::data_contract::accessors::v0::DataContractV0Getters;
+use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters;
+use dpp::version::PlatformVersion;
+use grovedb::TransactionArg;
+
+#[cfg(feature = "server")]
+impl Drive {
+    /// Joint count-and-sum no-prove dispatcher.
+    ///
+    /// Consumes a [`DocumentAverageRequest`] with `prove = false` and
+    /// produces a [`DocumentAverageResponse`] in the appropriate shape
+    /// (`Aggregate { count, sum }` for non-grouped modes,
+    /// `Entries(Vec<AverageEntry>)` for grouped modes).
+    ///
+    /// The dispatcher returns `Proof(_)`-shape responses only when
+    /// invoked via [`Drive::execute_document_average_request`]'s
+    /// `prove = true` arm; the no-prove path here never produces proof
+    /// bytes.
+    ///
+    /// # Routing
+    ///
+    /// Reuses sum's versioned routing table via
+    /// [`detect_sum_mode_from_inputs`] with the request's
+    /// [`AverageMode`] converted 1:1 to [`SumMode`] (the two enums are
+    /// structurally identical — same four variants). `prove = false`
+    /// is passed unconditionally because the prove path never reaches
+    /// this function. The resolved [`DocumentSumMode`] is one of three
+    /// no-prove values: `Total`, `PerInValue`, or `RangeNoProof`; the
+    /// four prove-mode values are unreachable. The match arm therefore
+    /// returns `CorruptedCodeExecution` on those branches rather than
+    /// silently misbehaving — if the routing table is ever extended
+    /// with a new prove-only mode that accidentally fires here, we
+    /// want to fail loudly.
+    pub fn execute_document_count_and_sum_request(
+        &self,
+        request: DocumentAverageRequest,
+        transaction: TransactionArg,
+        platform_version: &PlatformVersion,
+    ) -> Result<DocumentAverageResponse, Error> {
+        // Convert AverageMode → SumMode (1:1 by construction); sum's
+        // routing table is the single source of truth for the
+        // `(where_clauses × mode × prove=false)` triple. Forking a
+        // parallel "count_and_sum mode" table here is what #3687
+        // explicitly warned against — PR #3661 caught one drift bug
+        // (count's `GroupByCompound` row had diverged from sum's),
+        // and the joint executor's existence is itself the reason a
+        // single source of truth is now mandatory.
+        let sum_mode = match request.mode {
+            AverageMode::Aggregate => SumMode::Aggregate,
+            AverageMode::GroupByIn => SumMode::GroupByIn,
+            AverageMode::GroupByRange => SumMode::GroupByRange,
+            AverageMode::GroupByCompound => SumMode::GroupByCompound,
+        };
+
+        let resolved_mode =
+            detect_sum_mode_from_inputs(&request.where_clauses, sum_mode, false, platform_version)?;
+
+        let contract_id = request.contract.id().to_buffer();
+        let document_type_name = request.document_type.name().to_string();
+        let where_clauses = request.where_clauses;
+        let sum_property = request.sum_property;
+        let order_by_ascending = request
+            .order_clauses
+            .first()
+            .map(|c| c.ascending)
+            .unwrap_or(true);
+
+        match resolved_mode {
+            DocumentSumMode::Total => self.execute_document_count_and_sum_total_no_proof(
+                contract_id,
+                request.document_type,
+                document_type_name,
+                where_clauses,
+                sum_property,
+                transaction,
+                platform_version,
+            ),
+            DocumentSumMode::PerInValue => {
+                let options = RangeSumOptions {
+                    return_distinct_sums_in_range: false,
+                    carrier_outer_limit: None,
+                    left_to_right: order_by_ascending,
+                };
+                self.execute_document_count_and_sum_per_in_value_no_proof(
+                    contract_id,
+                    request.document_type,
+                    document_type_name,
+                    where_clauses,
+                    sum_property,
+                    options,
+                    transaction,
+                    platform_version,
+                )
+            }
+            DocumentSumMode::RangeNoProof => {
+                // Distinct flag set for the GroupByRange /
+                // GroupByCompound shapes (per-distinct-value entries);
+                // cleared for the Aggregate / GroupByIn shapes (single
+                // folded pair). Mirrors sum's dispatcher.
+                let return_distinct = matches!(
+                    request.mode,
+                    AverageMode::GroupByRange | AverageMode::GroupByCompound
+                );
+                let options = RangeSumOptions {
+                    return_distinct_sums_in_range: return_distinct,
+                    carrier_outer_limit: None,
+                    left_to_right: order_by_ascending,
+                };
+                let response = self.execute_document_count_and_sum_range_no_proof(
+                    contract_id,
+                    request.document_type,
+                    document_type_name,
+                    where_clauses,
+                    sum_property,
+                    options,
+                    transaction,
+                    platform_version,
+                )?;
+                // Sum's dispatcher applies a second-stage shape on the
+                // RangeNoProof path: `Aggregate` mode collapses to a
+                // single value, every other mode (GroupByIn /
+                // GroupByRange / GroupByCompound) returns `Entries`.
+                // GroupByIn + range specifically returns
+                // `Entries(vec![one_entry])` with the In axis folded
+                // (sum's executor itself emits exactly one entry in
+                // that case — see its `execute_range_sum_no_proof`
+                // compound-summed branch). The joint executor returns
+                // `Aggregate { count, sum }` for the !distinct shape
+                // because it has both metrics on hand; we re-shape into
+                // `Entries(vec![one_entry])` here when the caller asked
+                // for a non-Aggregate mode so the wire shape matches
+                // what an independent sum + count GroupByIn dispatch
+                // would produce. Without this re-shape, GroupByIn +
+                // range AVG silently changes wire shape from `Entries`
+                // (pre-#3687) to `Aggregate`.
+                // Strict mode×shape pairing. Every legal combination
+                // is named explicitly; any other pairing surfaces as
+                // `CorruptedCodeExecution` rather than silently
+                // forwarding a wrong-shape response. A future executor
+                // change that emits, say, `Entries` from an
+                // `AverageMode::Aggregate` request would fail loudly
+                // here instead of leaking the wrong wire shape to the
+                // gRPC handler.
+                match (request.mode, response) {
+                    (AverageMode::Aggregate, resp @ DocumentAverageResponse::Aggregate { .. }) => {
+                        Ok(resp)
+                    }
+                    (AverageMode::GroupByIn, DocumentAverageResponse::Aggregate { count, sum }) => {
+                        Ok(DocumentAverageResponse::Entries(vec![
+                            super::super::drive_document_average_query::AverageEntry {
+                                in_key: None,
+                                key: Vec::new(),
+                                count: Some(count),
+                                sum: Some(sum),
+                            },
+                        ]))
+                    }
+                    (
+                        AverageMode::GroupByRange | AverageMode::GroupByCompound,
+                        resp @ DocumentAverageResponse::Entries(_),
+                    ) => Ok(resp),
+                    (mode, _) => Err(Error::Drive(
+                        crate::error::drive::DriveError::CorruptedCodeExecution(match mode {
+                            AverageMode::Aggregate => {
+                                "execute_document_count_and_sum_request: \
+                                     RangeNoProof executor emitted a non-Aggregate \
+                                     response for AverageMode::Aggregate — joint \
+                                     range executor's shape contract violated"
+                            }
+                            AverageMode::GroupByIn => {
+                                "execute_document_count_and_sum_request: \
+                                     RangeNoProof executor emitted a non-Aggregate \
+                                     response for AverageMode::GroupByIn — the \
+                                     in-axis fold should yield a single (count, sum) \
+                                     pair the dispatcher re-wraps as Entries"
+                            }
+                            AverageMode::GroupByRange | AverageMode::GroupByCompound => {
+                                "execute_document_count_and_sum_request: \
+                                     RangeNoProof executor emitted a non-Entries \
+                                     response for a distinct grouped mode — joint \
+                                     range executor's shape contract violated"
+                            }
+                        }),
+                    )),
+                }
+            }
+            // The four prove-mode resolutions are unreachable here —
+            // the dispatcher passes `prove = false` to the routing
+            // table, and sum's v0 table has no row that maps a
+            // `prove=false` input to any of these four resolutions.
+            // Surface as `CorruptedCodeExecution` so a future routing-
+            // table change that breaks this assumption fails loudly
+            // rather than producing a runtime panic.
+            DocumentSumMode::RangeProof
+            | DocumentSumMode::RangeDistinctProof
+            | DocumentSumMode::PointLookupProof
+            | DocumentSumMode::RangeAggregateCarrierProof => Err(Error::Drive(
+                crate::error::drive::DriveError::CorruptedCodeExecution(
+                    "execute_document_count_and_sum_request: sum's routing table \
+                         resolved a prove-only mode from a `prove = false` request — \
+                         this indicates the routing table has been extended without \
+                         updating the joint dispatcher's match arms",
+                ),
+            )),
+        }
+    }
+}

packages/rs-drive/src/query/drive_document_count_and_sum_query/executors/mod.rs

@@ -0,0 +1,21 @@
+//! Per-`DocumentSumMode` joint count-and-sum no-prove executors.
+//!
+//! Mirrors [`crate::query::drive_document_sum_query::executors`]
+//! one-to-one for the no-prove subset of modes — the prove modes
+//! (`RangeProof`, `RangeDistinctProof`, `PointLookupProof`,
+//! `RangeAggregateCarrierProof`) are unreachable here because the AVG
+//! dispatcher routes prove-true requests through
+//! [`Drive::execute_document_average_prove`] before this module is
+//! consulted. The three no-prove modes are:
+//!
+//! - [`total`] — `DocumentSumMode::Total` (empty-where +
+//!   `documents_summable + documents_countable` fast path AND
+//!   Equal/In-fully-covered point lookup).
+//! - [`per_in_value`] — `DocumentSumMode::PerInValue` (Equal/In on a
+//!   non-range covered index, emitting one entry per In branch).
+//! - [`range_no_proof`] — `DocumentSumMode::RangeNoProof` (range and
+//!   distinct shapes over a PCPS-eligible index).
+
+pub mod per_in_value;
+pub mod range_no_proof;
+pub mod total;