Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions BUNDLE_API.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ that does not list `str(BUNDLE_API_VERSION)` is rejected at install time with
| Manifest schema (`extension.json` / `[tool.langflow.extension]`) | `lfx.extension.manifest.ExtensionManifest` |
| `BundleRef` (one entry in optional `bundles[]`; bundle names must be unique) | `lfx.extension.manifest.BundleRef` |
| `ProviderManifestEntry` (one entry in the optional `providers[]`) | `lfx.extension.manifest.ProviderManifestEntry` |
| `IntegrationProvider` entries in optional `integrations[]` | `lfx.integrations.IntegrationProvider` |
| `IntegrationManifestRef` entries in optional `integrations[]` | `lfx.extension.manifest.IntegrationManifestRef` |
| `IntegrationCapabilityManifest` (bundle-owned versioned provider catalog) | `lfx.integrations.IntegrationCapabilityManifest` |
| `LfxCompat` (declared as `manifest.lfx`) | `lfx.extension.manifest.LfxCompat` |
| `BUNDLE_API_VERSION` (the integer this lfx ships) | `lfx.extension.manifest` |
| `EXTENSION_SCHEMA_URL` / `SCHEMA_VERSION` | `lfx.extension.manifest` |
Expand All @@ -105,6 +106,7 @@ Component IDs at runtime are `ext:<bundle>:<Class>@<slot>`.
| `discover_inline_bundles()` | `lfx.extension.loader` |
| `discover_installed_extensions()` / `discover_seed_extensions()` / `discover_all_extensions()` | `lfx.extension.discovery` |
| `LoadedComponent` | `lfx.extension.loader` (frozen dataclass; what the registry stores) |
| `LoadedIntegration` | `lfx.extension.loader` (validated capability metadata for discovery and policy) |
| `LoadResult` | `lfx.extension.loader` |
| `SLOT_OFFICIAL` / `SLOT_EXTRA` | `lfx.extension.loader` |

Expand All @@ -113,8 +115,8 @@ Component IDs at runtime are `ext:<bundle>:<Class>@<slot>`.
| Symbol | Source |
| --- | --- |
| `reload_bundle(registry, bundle_name)` | `lfx.extension.reload` |
| `BundleRegistry` | `lfx.extension.bundle_registry` |
| `BundleRecord` | `lfx.extension.bundle_registry` |
| `BundleRegistry` (`list_components()`, `list_integrations()`) | `lfx.extension.bundle_registry` |
| `BundleRecord` (components plus validated integration metadata) | `lfx.extension.bundle_registry` |
| `ReloadInProgressError` | `lfx.extension.bundle_registry` |
| `POST /api/v1/extensions/{id}/bundles/{name}/reload` | `langflow.api.v1.extensions` |

Expand Down Expand Up @@ -205,6 +207,19 @@ the deserialize half is covered by

### v0 (this release)

- **Bundle-owned integration capability manifests (additive).**
`ExtensionManifest.integrations[]` now carries `IntegrationManifestRef`
values (`provider_id`, owning `bundle`, relative JSON `path`). The referenced
`IntegrationCapabilityManifest` is versioned with `schema_version=1` and
declares authentication profiles plus executable actions, required and
conditional scopes, policy keys, substrate, maturity, deployment contexts,
risk, and execution targets. The loader validates bundle ownership and
provider identity and exposes the parsed catalog as `LoadedIntegration` in
`LoadResult.integrations` and retains it in `BundleRecord.integrations` for
process-wide discovery and policy reads through
`BundleRegistry.list_integrations()`. Manifests that omit `integrations`
still load with an empty list; `BUNDLE_API_VERSION` remains `1`.

- Initial surface enumerated above. Frozen as `BUNDLE_API_VERSION = 1`.
- Added the provider-neutral connection-reference, resolver, capability,
integration-error, and telemetry contracts used by dedicated integration
Expand Down
16 changes: 11 additions & 5 deletions design/dedicated-integrations/connection-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,8 +273,8 @@ dataclass (not an `Exception`; loader-specific code namespace).

## 8. Capability metadata types shared with INT-3

**Decision: frozen Pydantic models in `lfx/integrations/capabilities.py`; reserve `ExtensionManifest.integrations`
now.**
**Decision: frozen Pydantic models in `lfx/integrations/capabilities.py`; `ExtensionManifest.integrations`
references bundle-owned, versioned capability manifests.**

- `IntegrationProvider{provider_id, display_name, icon, auth_profiles: tuple[OAuthProfile, ...], capabilities,
docs_url}`. A provider may expose more than one named profile; Slack has `slack-user-oauth` and
Expand All @@ -287,7 +287,9 @@ now.**
`oauth_client_type_by_context`; profiles may omit an unsupported context. This represents both Tauri public-client
loopback and Microsoft's `{tenant}` authority without pretending one provider-wide auth object fits every action.
- `IntegrationCapability{id, display_name, auth_profile_id, identity, required_scopes, conditional_scopes,
risk: read | write | destructive, component_ref, mcp_tool}`. `ConditionalScopeRequirement{scope, role,
policy_keys, substrate: sdk | rest | mcp, maturity: ga | preview | developer_preview | beta | deprecated,
deployment_contexts, risk: read | write | destructive, component_ref, mcp_tool}`.
`ConditionalScopeRequirement{scope, role,
condition}` preserves `optional` versus `alternative`; `ScopeCondition{kind: input_present | input_truthy,
input}` is evaluated against the action's declared input schema. The matrix checker rejects a condition that
names a missing input. `ScopeSet.covers(capability, inputs, granted) -> missing` first activates conditional
Expand All @@ -296,8 +298,12 @@ now.**
- Capability ids are the matrices' `action_id` values. `required` rows become `required_scopes`; `optional` and
`alternative` rows become `conditional_scopes` without losing their role or predicate. The capability's
`auth_profile_id` and `identity` must match the selected connection before scope coverage is evaluated.
- `ExtensionManifest.integrations: tuple[IntegrationProvider, ...] = ()` is added as an optional field (additive;
`manifest.py` is in the changelog gate); loader wiring is INT-3.
- `ExtensionManifest.integrations: tuple[IntegrationManifestRef, ...] = ()`, where each reference declares
`{provider_id, bundle, path}` and `path` is relative to the named bundle. The referenced
`IntegrationCapabilityManifest` extends `IntegrationProvider` with `schema_version: Literal[1]`. Validation
rejects missing, malformed, provider-mismatched, or bundle-escaping files; the loader exposes successful records
through `LoadResult.integrations` as `LoadedIntegration` values and retains them in the process-wide
`BundleRegistry` for discovery and policy consumers.

Rejected: reusing `ProviderManifestEntry` (model-provider registry semantics would route integrations into
model-provider policy); JSON-schema only (the resolver and the picker need the same scope math in Python).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://schemas.langflow.org/design/dedicated-integrations/capability-matrix-v1.json",
"title": "Dedicated Integrations wave-1 capability matrix",
"description": "One provider's wave-1 capability matrix for the INT-1 discovery gate. Hand-written for the gate; INT-3 lifts the enums into Pydantic under src/lfx/src/lfx/extension/ and regenerates this file. The enum values here are asserted equal to VALID_VALUES in scripts/ci/check_capability_matrices.py.",
"description": "One provider's wave-1 capability matrix for the discovery gate. This remains the historical planning record; runtime capability manifests use the matching Pydantic enums in src/lfx/src/lfx/integrations/capabilities.py. Tests and scripts/ci/check_capability_matrices.py guard enum parity.",
"type": "object",
"additionalProperties": false,
"required": [
Expand Down
69 changes: 68 additions & 1 deletion docs/docs/Lfx/extensions-manifest.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Use the `$schema` reference in your manifest so editors can autocomplete and val
| `bundles` | array | no | [Bundle list](#bundles). v0 accepts at most one bundle; omit it for a provider-only extension. |
| `providers` | array | no | [Model providers](#providers) contributed to the unified model-provider registry. |
| `capabilities` | object | no | [Optional capability flags](#capabilities). Defaults to all-false. |
| `integrations` | array | no | [Bundle-owned integration capability manifests](#integrations). Defaults to an empty list. |
| `$schema` | string | no | Optional pointer to this JSON Schema; editors use it for autocomplete. |

`additionalProperties: false` — any field not listed here is rejected with a typed error. Reserved names (`services`, `routes`, `hooks`, `starterProjects`, `userConfig`) are documented under [Deferred fields](#deferred-fields) and surface a more specific error code.
Expand Down Expand Up @@ -142,6 +143,72 @@ Optional. Defaults to `{ "requiresCredentials": false }`.

Additional capability keys are rejected with `extra="forbid"` so a misspelled key surfaces immediately rather than silently turning a feature off.

## `integrations`

Integration bundles publish provider capabilities in a separate, versioned JSON file owned by the bundle. The extension manifest points to that file:

```json
"integrations": [
{
"provider_id": "google",
"bundle": "google",
"path": "capabilities.v1.json"
}
]
```

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `provider_id` | string | yes | Stable lowercase provider key used by discovery, policy, and connection resolution. |
| `bundle` | string | yes | Name of the bundle that owns the capability manifest. It must match an entry in `bundles[]`. |
| `path` | string | yes | JSON file path relative to that bundle's directory. Absolute paths, `..`, and symlink escapes are rejected. |

The referenced file uses this shape:

```json
{
"schema_version": 1,
"provider_id": "google",
"display_name": "Google Workspace",
"auth_profiles": [
{
"id": "user",
"kind": "oauth2_authorization_code",
"identity": "user_delegated",
"supports_pkce": true,
"supports_refresh": true
}
],
"capabilities": [
{
"id": "google.drive.files.search",
"display_name": "Drive: Search Files",
"auth_profile_id": "user",
"identity": "user_delegated",
"required_scopes": ["drive.file"],
"policy_keys": ["integrations.google.drive.search"],
"substrate": "sdk",
"maturity": "ga",
"deployment_contexts": ["hosted", "self_managed", "desktop", "headless"],
"risk": "read",
"component_ref": "GoogleDriveSearchComponent"
}
]
}
```

`schema_version` is currently `1`. Authentication modes are declared by `auth_profiles[].kind`; actions reference a profile through `auth_profile_id`. Each action declares scopes, policy keys, execution substrate (`sdk`, `rest`, or `mcp`), maturity, supported deployment contexts, risk, and at least one execution target (`component_ref` or `mcp_tool`). Unknown fields are rejected throughout.

During loading, each valid file is exposed as a `LoadedIntegration` in `LoadResult.integrations` and retained in `BundleRecord.integrations`. Discovery and policy code can consume the parsed `capability_manifest` from the load result or call `BundleRegistry.list_integrations()` for the process-wide snapshot. Extensions that omit `integrations` continue to load with an empty list.

After adding an integration manifest on a release branch, synchronize bundle dependency floors with the repository script instead of editing `pyproject.toml` by hand:

```bash
uv run python scripts/ci/sync_bundle_lfx_pin.py 1.13.0
```

The command is idempotent and applies the canonical `lfx>=1.13.0.dev0,<2.0.0` floor to bundles that need an update.

## Deferred fields

The schema strips these names from the published `properties` map but reserves them via [`x-deferred-fields`](https://schemas.langflow.org/extension/v1.json), so a manifest that sets one gets a specific error code instead of the generic "additional property" message.
Expand Down Expand Up @@ -182,7 +249,7 @@ The loader and validator both emit typed errors keyed by the manifest field that

| Code | Cause |
| --- | --- |
| `manifest-invalid` | Schema validation failed; the message names the field. |
| `manifest-invalid` | The extension manifest or a referenced integration capability manifest fails schema validation. |
| `manifest-not-found` | No `extension.json` and no `[tool.langflow.extension]` section at the extension root. |
| `version-constraint-unsatisfied` | `lfx.compat` does not include this Langflow's `BUNDLE_API_VERSION`. |
| `field-deferred-in-this-milestone` | A reserved field was set to a non-null value. |
Expand Down
6 changes: 6 additions & 0 deletions src/lfx/src/lfx/extension/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
SLOT_EXTRA,
SLOT_OFFICIAL,
LoadedComponent,
LoadedIntegration,
LoadResult,
discover_inline_bundles,
filter_component_entry_points,
Expand All @@ -90,6 +91,7 @@
SCHEMA_VERSION,
BundleRef,
ExtensionManifest,
IntegrationManifestRef,
LfxCompat,
ManifestSource,
load_manifest,
Expand Down Expand Up @@ -135,6 +137,7 @@
"SCHEMA_VERSION": "manifest",
"BundleRef": "manifest",
"ExtensionManifest": "manifest",
"IntegrationManifestRef": "manifest",
"LfxCompat": "manifest",
"ManifestSource": "manifest",
"load_manifest": "manifest",
Expand All @@ -155,6 +158,7 @@
"SLOT_OFFICIAL": "loader",
"LoadResult": "loader",
"LoadedComponent": "loader",
"LoadedIntegration": "loader",
"discover_inline_bundles": "loader",
"filter_component_entry_points": "loader",
"filter_plugin_entry_points": "loader",
Expand Down Expand Up @@ -240,10 +244,12 @@ def __dir__() -> list[str]:
"ExtensionManifest",
"ExtensionRegistry",
"InitOptions",
"IntegrationManifestRef",
"LfxCompat",
"LoadResult",
"LoadStatus",
"LoadedComponent",
"LoadedIntegration",
"ManifestSource",
"MigrationEntry",
"MigrationReport",
Expand Down
11 changes: 10 additions & 1 deletion src/lfx/src/lfx/extension/bundle_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ class set even after Stage 3 has flipped the registry's internal pointer.
from pathlib import Path
from typing import Literal

from lfx.extension.loader import LoadedComponent
from lfx.extension.loader import LoadedComponent, LoadedIntegration


# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -96,6 +96,7 @@ class BundleRecord:
extension_version: str
slot: Literal["official", "extra"]
components: tuple[LoadedComponent, ...] = ()
integrations: tuple[LoadedIntegration, ...] = ()
distribution: str | None = None
source_path: Path | None = None
# Provenance: True for manifest-less lfx.bundles metapackage providers.
Expand Down Expand Up @@ -168,6 +169,14 @@ def list_components(self) -> list[LoadedComponent]:
out.extend(snap[name].components)
return out

def list_integrations(self) -> list[LoadedIntegration]:
"""Flatten validated integration metadata for discovery and policy consumers."""
snap = self.snapshot()
out: list[LoadedIntegration] = []
for name in sorted(snap):
out.extend(snap[name].integrations)
return out

# -- write paths ---------------------------------------------------------

@contextmanager
Expand Down
93 changes: 93 additions & 0 deletions src/lfx/src/lfx/extension/integration_manifest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
"""Resolve and validate bundle-owned integration capability manifests."""

from __future__ import annotations

import json
from dataclasses import dataclass
from typing import TYPE_CHECKING

from pydantic import ValidationError

from lfx.extension._paths import is_within
from lfx.extension.errors import ExtensionError
from lfx.integrations.capabilities import IntegrationCapabilityManifest

if TYPE_CHECKING:
from pathlib import Path

from lfx.extension.manifest import IntegrationManifestRef


@dataclass(frozen=True)
class ResolvedIntegrationManifest:
"""A validated capability catalog paired with its canonical source path."""

path: Path
manifest: IntegrationCapabilityManifest


def resolve_integration_manifest(
bundle_root: Path,
reference: IntegrationManifestRef,
) -> tuple[ResolvedIntegrationManifest | None, ExtensionError | None]:
"""Load one reference while enforcing bundle ownership and provider identity."""
try:
resolved_root = bundle_root.resolve(strict=True)
candidate = (resolved_root / reference.path).resolve(strict=False)
except (OSError, RuntimeError, ValueError) as exc:
return None, ExtensionError(
code="manifest-unreadable",
message=f"Could not resolve integration capability manifest: {exc}",
location=reference.path,
content=reference.path,
hint="Check the capability-manifest path and bundle permissions.",
)

if not is_within(candidate, resolved_root):
return None, ExtensionError(
code="path-escape",
message=(
f"Integration capability-manifest path {reference.path!r} resolves outside "
f"its owning bundle {reference.bundle!r}."
),
location=f"integrations[{reference.provider_id}].path",
content=reference.path,
hint="Move the capability manifest inside the bundle directory.",
)

try:
raw = candidate.read_text(encoding="utf-8")
except (OSError, UnicodeError, ValueError) as exc:
return None, ExtensionError(
code="manifest-unreadable",
message=f"Could not read integration capability manifest: {exc}",
location=str(candidate),
content=reference.path,
hint="Create the referenced JSON file inside the bundle and check its permissions.",
)

try:
payload = json.loads(raw)
manifest = IntegrationCapabilityManifest.model_validate(payload)
except (json.JSONDecodeError, TypeError, ValidationError) as exc:
return None, ExtensionError(
code="manifest-invalid",
message=f"Integration capability manifest is invalid: {exc}",
location=str(candidate),
content=reference.path,
hint="Fix the referenced capability manifest so it matches the versioned integration schema.",
)

if manifest.provider_id != reference.provider_id:
return None, ExtensionError(
code="manifest-invalid",
message=(
f"Integration provider {reference.provider_id!r} references a capability manifest for "
f"{manifest.provider_id!r}."
),
location=str(candidate),
content=manifest.provider_id,
hint="Make provider_id identical in extension.json and the capability manifest.",
)

return ResolvedIntegrationManifest(path=candidate, manifest=manifest), None
2 changes: 2 additions & 0 deletions src/lfx/src/lfx/extension/loader/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@
SLOT_OFFICIAL,
SLOT_VALUES,
LoadedComponent,
LoadedIntegration,
LoadResult,
)

Expand All @@ -89,6 +90,7 @@
"SLOT_VALUES",
"LoadResult",
"LoadedComponent",
"LoadedIntegration",
"discover_inline_bundles",
"filter_component_entry_points",
"filter_plugin_entry_points",
Expand Down
Loading
Loading