Chargeback 0.5.1: type/namespace filter zip and E2E

CHANGELOG.md

@@ -10,6 +10,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## Integration Releases
 
+### [0.5.0] - 2026-06-05
+
+#### Changed
+
+- Billing Components Overview and Usage & Cost Allocation dashboards: legacy options-list controls replaced with chained ES|QL multi-select variable controls (`MV_CONTAINS` filtering).
+- Kibana minimum version: **9.4.0+** (Elasticsearch remains **9.2.0+**). **Kibana 9.3 is not supported** — Billing and Usage dashboards fail with `No embeddable factory found for type: vis`. Transform `fleet_transform_version` and pipeline references bumped to `0.5.0` (pipeline logic unchanged).
+
+#### Upgrade notes
+
+When upgrading from 0.4.x:
+1. Upgrade Kibana to **9.4.0+** before installing 0.5.0 (9.3 is unsupported).
+2. Upgrade the Fleet package to **0.5.0**; replace duplicate Billing/Usage dashboard saved objects if needed.
+
+---
+
 ### [0.4.0] - 2026-06-01
 
 #### Added

integration/Instructions.md

@@ -14,12 +14,14 @@ To install the Chargeback integration, please follow these steps:
 
 See [Requirements](README.md#requirements) for details.
 
-**Key prerequisite for 0.4.0:** The [Elasticsearch integration](https://www.elastic.co/docs/reference/integrations/elasticsearch/) must be **actively running** on all monitored deployments with both **index stats** and **node stats** datasets enabled. Node stats are required for the realized cost utilization score. Without them, utilization defaults to 100% and no discount is applied.
+**Key prerequisite for 0.4.0+:** The [Elasticsearch integration](https://www.elastic.co/docs/reference/integrations/elasticsearch/) must be **actively running** on all monitored deployments with both **index stats** and **node stats** datasets enabled. Node stats are required for the realized cost utilization score. Without them, utilization defaults to 100% and no discount is applied.
+
+**Key prerequisite for 0.5.0+:** Kibana **9.4.0+**. **Kibana 9.3 is not supported** — Billing and Usage dashboards fail to render with `No embeddable factory found for type: vis` and do not support GA ES|QL multi-select variable controls.
 
 ### 2. Upload ZIP File: 
 
-- Asset: [`chargeback-0.4.0.zip`](assets/0.4.0/chargeback-0.4.0.zip)
-- Browse to Integrations, and click on `+ Create new integration`
+- Asset: [`chargeback-0.5.1.zip`](assets/0.5.1/chargeback-0.5.1.zip)
+- Browse to Integrations, and select **+ Create new integration**
 
 ![alt text](assets/img/CreateNewIntegration.png)
 
@@ -31,7 +33,7 @@ See [Requirements](README.md#requirements) for details.
 
 Starting from version 0.2.8, all Chargeback transforms are configured to auto-start upon installation. You no longer need to manually start the transforms.
 
-**Starting from version 0.2.10** (current: **v0.4.0**), the `chargeback_conf_lookup` index is automatically created via a bootstrap transform during installation. No manual setup is required! The transform creates the index with default configuration:
+**Starting from version 0.2.10** (current: **v0.5.1**), the `chargeback_conf_lookup` index is automatically created via a bootstrap transform during installation. No manual setup is required! The transform creates the index with default configuration:
 - **Chargeable unit rate:** 0.85 EUR
 - **Weights:** indexing=20, query=20, storage=40
 - **Date range:** 2010-01-01 to 2046-12-31
@@ -57,6 +59,16 @@ To upgrade the integration, do the following:
 - Upload the new asset (ZIP) file to Kibana.
 - Transforms will auto-start (from version 0.2.8 onwards).
 
+**Upgrading from 0.5.0 to 0.5.1:**
+- Upload `chargeback-0.5.1.zip` (Kibana **9.4.0+**).
+- Usage transforms reinstall with `fleet_transform_version: 0.5.1` and add `ds_type` / `ds_namespace`. Reset those transforms if existing lookup docs lack the new fields.
+- Usage & Cost Allocation gains data stream type and namespace controls. Delete duplicate dashboard saved objects if they appear after upgrade.
+
+**Upgrading from 0.4.x to 0.5.0:**
+- Upgrade Kibana to **9.4.0+**, then upload `chargeback-0.5.0.zip`. Do not install on Kibana 9.3.
+- Transforms are reinstalled with `fleet_transform_version: 0.5.0` (pipeline logic unchanged). No lookup index recreation is required.
+- Billing and Usage dashboards are replaced with ES|QL multi-select variable controls. Delete duplicate dashboard saved objects if they appear after upgrade.
+
 **Upgrading from 0.3.x to 0.4.0:**
 - Upload `chargeback-0.4.0.zip`. Two new transforms (`billing_realized_pool`, `cluster_capacity_utilization`) are created and auto-started.
 - The old `[Chargeback] Cost and Consumption breakdown` dashboard is replaced by three new dashboards (`[Chargeback] Billing Components Overview`, `[Chargeback] Usage & Cost Allocation`, and `[Chargeback] Configuration`). If Kibana does not replace the old dashboard automatically, delete it and re-import from **Saved Objects**.
@@ -81,7 +93,7 @@ If the dashboard is empty after install, see **[docs/troubleshooting.md](docs/tr
 
 ## Configuration
 
-Configuration values are stored in the `chargeback_conf_lookup` index, which is automatically created by version 0.2.10+ (current: **v0.4.0**). The dashboard automatically applies the correct configuration based on the billing date falling within the `conf_start_date` and `conf_end_date` range.
+Configuration values are stored in the `chargeback_conf_lookup` index, which is automatically created by version 0.2.10+ (current: **v0.5.0**). The dashboard automatically applies the correct configuration based on the billing date falling within the `conf_start_date` and `conf_end_date` range.
 
 ### Update the default configuration:
 

integration/README.md

@@ -2,7 +2,7 @@
 
 ## Version
 
-Chargeback integration: 0.4.0
+Chargeback integration: 0.5.1
 
 ## Dependencies
 
@@ -12,7 +12,8 @@ This process must be set up on the **Monitoring cluster**, where all monitoring
 
 To use this integration, the following prerequisites must be met:
 
-- The monitoring cluster, where this integration is installed, must be on version 9.2.0+ due to its use of (smart) [ES|QL LOOKUP JOIN](https://www.elastic.co/docs/reference/query-languages/esql/esql-lookup-join).
+- The monitoring cluster, where this integration is installed, must be on Elasticsearch version 9.2.0+ due to its use of (smart) [ES|QL LOOKUP JOIN](https://www.elastic.co/docs/reference/query-languages/esql/esql-lookup-join).
+- Kibana version **9.4.0+** is required for integration **0.5.0** and later. **Kibana 9.3 is not supported** — Billing and Usage dashboards fail to render (`No embeddable factory found for type: vis`) and lack GA support for ES|QL multi-select variable controls.
 - The [**Elasticsearch Service Billing**](https://www.elastic.co/docs/reference/integrations/ess_billing/) integration (v1.7.0+) must be installed and running.
 - The [**Elasticsearch**](https://www.elastic.co/docs/reference/integrations/elasticsearch/) integration (v1.16.0+) must be **installed and actively running** on all monitored deployments, with the following datasets enabled:
   - **Index stats** — required for tier and data stream cost allocation. The `logs-elasticsearch.index_pivot-default-{VERSION}` transform must be running to aggregate these into `monitoring-indices`.
@@ -29,7 +30,9 @@ This integration must be installed on the **Monitoring cluster** where the above
 | 0.2.10 - 0.2.x | 9.2.0+ | 1.7.0+ | Requires ESS Billing 1.7.0 features |
 | 0.3.0 | 9.2.0+ | 1.7.0+ | Chargeable units schema (breaking change from 0.2.x) |
 | 0.3.1 - 0.3.2 | 9.2.0+ | 1.7.0+ | Field renames, deployment_tags fix, explicit lookup mappings |
-| 0.4.0+ | 9.2.0+ | 1.7.0+ | Realized cost model, SKU classification, three-dashboard split |
+| 0.4.0 | 9.2.0+ | 1.7.0+ | Realized cost model, SKU classification, three-dashboard split |
+| 0.5.0+ | 9.4.0+ (Kibana), 9.2.0+ (ES). **Not 9.3** | 1.7.0+ | ES|QL multi-select variable controls; Kibana 9.3 fails with `No embeddable factory found for type: vis` |
+| 0.5.1 | 9.4.0+ (Kibana), 9.2.0+ (ES). **Not 9.3** | 1.7.0+ | `ds_type` / `ds_namespace` on usage path; Usage dashboard type and namespace controls |
 
 ## Setup instructions
 
@@ -58,8 +61,8 @@ The first layer of processing that we do, is eight transforms:
 **Usage transforms** (from monitoring indices):
 - `cluster_deployment_contribution` — indexing, querying, and storage metrics per deployment per day.
 - `cluster_tier_contribution` — same metrics split by data tier.
-- `cluster_datastream_contribution` — same metrics split by data stream.
-- `cluster_tier_and_ds_contribution` — same metrics split by both tier and data stream.
+- `cluster_datastream_contribution` — same metrics split by data stream; usage pipeline sets `ds_type` and `ds_namespace`.
+- `cluster_tier_and_ds_contribution` — same metrics split by both tier and data stream (includes `ds_type` and `ds_namespace`).
 
 ![Transforms](assets/img/Transforms.png)
 
@@ -94,10 +97,16 @@ Answers: *what did we spend and where did it go?*
 
 Answers: *which data streams and tiers drive cost, and how efficiently are we using capacity?*
 
+Controls include deployment group, deployment name, data tier, full data stream name, **data stream type** (`ds_type`), and **data stream namespace** (`ds_namespace`).
+
 - **Data tiers / utilization** — provisioned capacity versus realized pool (`chargeable_pool = provisioned × util_score`), p95 heap and disk utilization.
 - **Data tier and data stream overview** — top-20 data streams by indexing / query / storage cost, blended cost totals, workload mix by tier.
 - **Data tier and data stream per day** — time-series cost breakdown (indexing, querying, storage, blended) by data stream and tier, including percentage share panels.
 
+### Shared deployments and namespaces
+
+On a shared deployment, assign each team a unique [Fleet data stream namespace](https://www.elastic.co/docs/reference/fleet/data-streams) (`type-dataset-namespace`). Chargeback parses `datastream` into `ds_type` (first segment) and `ds_namespace` (last segment), with `other` when the name has no hyphen. Use the Usage dashboard type and namespace controls for team-level FinOps. A dataset control is not included in 0.5.1.
+
 ### [Chargeback] Configuration
 
 A standalone reference dashboard showing all active configuration values: conversion rate, date windows, blended cost weights, utilization score weights, and memory/storage cost split — each visualised as a percentage-stacked bar chart.
@@ -114,6 +123,32 @@ These alerting templates are automatically installed with the integration and ca
 
 **Important:** For alert rules 2 and 3, ensure that the Chargeback transforms are running before setting them up. These alerting rules query the lookup indices created by the transforms (`billing_cluster_cost_lookup`, `cluster_deployment_contribution_lookup`, etc.). If the transforms are not started, the alerts will not function correctly.
 
+## Version 0.5.1 Release Notes
+
+### Added
+
+- Usage ingest pipeline materializes `ds_type` and `ds_namespace` from each `datastream` name (Fleet `type-dataset-namespace` scheme; `other` fallback).
+- Usage & Cost Allocation dashboard: **Data stream type** and **Data stream namespace** ES|QL multi-select controls.
+- Docs for namespace-based ownership on shared deployments.
+
+### Upgrade from 0.5.0
+
+1. Upload `chargeback-0.5.1.zip` (Kibana **9.4.0+**).
+2. Reset usage transforms that write through the usage pipeline if existing lookup docs lack `ds_type` / `ds_namespace`.
+3. Replace Usage dashboard saved objects if duplicates appear after upgrade.
+
+## Version 0.5.0 Release Notes
+
+### Changed
+
+- **Billing and Usage dashboards** use chained ES|QL multi-select variable controls instead of legacy options-list controls. Panel queries filter with `MV_INTERSECTS` and a `__chargeback_unfiltered__` sentinel default.
+- Package `kibana.version` constraint raised to `^9.4.0`. **Kibana 9.3 is not supported** — dashboards fail with `No embeddable factory found for type: vis`. Transform `fleet_transform_version` and pipeline references bumped to `0.5.0` (pipeline logic unchanged).
+
+### Upgrade from 0.4.x
+
+1. Upgrade Kibana to **9.4.0+**, then upgrade the Fleet package to **0.5.0**. Do not install 0.5.0 on Kibana 9.3.
+2. Re-import or replace the Billing and Usage dashboard saved objects if duplicates appear after upgrade.
+
 ## Version 0.4.0 Release Notes
 
 ### Added

integration/docs/troubleshooting.md

@@ -244,10 +244,89 @@ The bundled dashboard is **Fleet/Kibana managed**—you cannot fix this by editi
 
 There is no supported workaround on **0.3.1** other than upgrading the package.
 
+## `x_content_parse_exception`: empty lists in ES|QL `params` (0.5.0 on Elasticsearch 9.4+)
+
+### Symptom
+
+Billing or Usage dashboard panels fail with:
+
+`Unexpected error from Elasticsearch: x_content_parse_exception - [esql/async_query] failed to parse field [params]`
+
+and messages like:
+
+`Empty lists are not allowed as named parameter values. Got parameter [dg_selected] with value [[]]`
+
+Kibana may also report **“N panels have been removed from the dashboard”** after a bad dashboard import or manual edit that strips ES|QL filter clauses or replaces ES|QL controls with legacy options-list controls.
+
+### Cause
+
+**0.5.0** Billing and Usage dashboards use chained **ES|QL multi-select variable controls** (`variable_type: multi_values`) with this filter pattern:
+
+```esql
+WHERE (?dg_selected IS NULL OR MV_CONTAINS(?dg_selected, deployment_group))
+```
+
+When no values are selected, Kibana sends **empty arrays** in the ES|QL request:
+
+```json
+"params": [{ "dg_selected": [] }, { "deployment_selected": [] }]
+```
+
+From **Elasticsearch 9.4** onward, **named ES|QL parameters cannot be `[]`**. Elasticsearch rejects the request at parse time ([elastic/elasticsearch#147448](https://github.com/elastic/elasticsearch/issues/147448), [PR #147748](https://github.com/elastic/elasticsearch/pull/147748)). The `?var IS NULL` branch never runs because the parameter value is `[]`, not SQL `NULL`.
+
+The Chargeback query pattern is correct. The bug is in **Kibana param serialization**: `esql_control_manager.ts` sets `value = selectedValues.map(...)` for multi-select (which is `[]` when nothing is selected), and `getNamedParams()` in `@kbn/esql-utils` forwards that value unchanged to Elasticsearch.
+
+**Do not “fix” this in the Chargeback package by:**
+
+- Removing `?variable` filters from panel ES|QL (breaks multi-select filtering; may drop panels on import)
+- Reverting to legacy `options_list_control` / `controlGroupInput` (removes chained tier / data stream / cost category / cost type filters)
+- Adding an `"All"` sentinel value (breaks multi-select UX)
+
+### Verified behaviour on Elasticsearch 9.4
+
+| Request param | Result with `?var IS NULL OR MV_CONTAINS(?var, field)` |
+|---------------|--------------------------------------------------------|
+| `[{"var": null}]` | Works — shows all rows (no filter) |
+| `[{"var": ["product"]}]` | Works — filters correctly |
+| `[{"var": []}]` | **400 parse error** (same error you see in Kibana) |
+| Param omitted entirely | **Unknown query parameter** error |
+
+### Workaround (until Kibana is patched)
+
+On each ES|QL control in the Billing / Usage dashboard, open the control and choose **Select all** so `selected_options` is populated with real values instead of `[]`. Save the dashboard if you want that state to persist.
+
+This is a UX workaround only. Fresh installs of **0.5.0** still ship with `"selected_options": []` on each control.
+
+### Upstream fix (Kibana)
+
+Kibana should send `null` (one param object per variable) when a `multi_values` control has no selection, matching the Agent Builder pattern in [kibana#256588](https://github.com/elastic/kibana/pull/256588). Minimal fix in `getNamedParams()` (`src/platform/packages/shared/kbn-esql-utils/src/utils/run_query.ts`):
+
+```typescript
+if (type === ESQLVariableType.MULTI_VALUES && Array.isArray(value) && value.length === 0) {
+  namedParams.push({ [key]: null });
+} else {
+  namedParams.push({ [key]: value });
+}
+```
+
+See also `scripts/DESIGN_ESQL_EMPTY_PARAMS.md` in this repository for full analysis.
+
+### If panels were removed from the dashboard
+
+Reinstall **0.5.0** dashboards from the integration package (do not hand-edit controls back to options-list). With `REPLACE_CHARGEBACK_DASHBOARD=1`:
+
+```bash
+REPLACE_CHARGEBACK_DASHBOARD=1 ./scripts/run_e2e_tests.sh
+```
+
+Or delete the Billing / Usage dashboard saved objects in **Stack Management → Saved Objects** and reinstall the Chargeback integration zip.
+
 ## Symptom → likely cause
 
 | Symptom | Likely cause |
 |---------|----------------|
+| `Empty lists are not allowed as named parameter values` on Billing/Usage (0.5.0, ES 9.4+) | Kibana sends `[]` for empty multi-select ES|QL controls; needs Kibana fix or **Select all** workaround |
+| “N panels have been removed from the dashboard” after upgrade | Corrupt dashboard import (stripped ES|QL filters or wrong control type); reinstall 0.5.0 dashboards from package |
 | All panels empty; Chargeback transforms “started” | `logs-elasticsearch.index_pivot-default-*` not started, or `monitoring-indices*` empty |
 | ESS Billing dashboard has data; Chargeback empty | `billing_cluster_cost` not finished first run (60m + 1h delay), transform failed, or `total_ecu` not > 0 |
 | `billing_cluster_cost_lookup` has docs; cost panels empty | `chargeback_conf_lookup` date range does not cover billing `@timestamp` |