Add documentation for team-based asset event filtering by vincbeck · Pull Request #65690 · apache/airflow

vincbeck · 2026-04-22T18:48:41Z

Create documentation for the new feature team-based asset event filtering as described in AIP-67.

You can also see the generated documentation as PDF here:

Asset Definitions — Airflow 3.3.0 Documentation.pdf
Multi-Team — Airflow 3.3.0 Documentation.pdf

Here is the description of the feature as described in the AIP:

While any DAG can produce events that are related to any data assets, the DAGs consuming the data assets will - by default - only receive events that are triggered in the same team or by an API call from a user that is recognized to belong to the same team by Auth Manager. DAG authors could specify a list of teams that should additionally be allowed to trigger the DAG (joined by the URI of the Data Asset) by adding a new asset parameter and the triggering will happend. Also this AIP depends on AIP-73 - Expanded Data Awareness - and the exact approach will piggiback on deailed implementation of Data ASsets. allow_triggering_by_teams = [ "team1", "team2" ]

I'll leave this PR open in draft while working on the feature. The goal is to merge this PR once the feature is implemented.

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)
Kiro (Claude)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

boring-cyborg · 2026-04-22T18:48:44Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

vincbeck · 2026-04-22T19:57:44Z

@eladkal If you're interested, I gave it a shot with the documentation-frst approach you described in the dev email list. So far I like it, it makes a lot of sense.

o-nikolas

Looking good! Just a few comments/suggestions.

Also can you upload a new PDF (or even better a screenshot of the rendered html doc) that has the table headers? They're cut off in the current PDF

o-nikolas · 2026-04-22T22:44:29Z

+.. versionadded:: 3.3.0
+
+When :doc:`Multi-Team mode </core-concepts/multi-team>` is enabled, asset events are filtered by team
+membership. By default, a consuming Dag only receives asset events produced by Dags within the same team.


Or global right? Or is that only when allow_teams is empty/not specified?

o-nikolas · 2026-04-22T22:45:28Z

+membership. By default, a consuming Dag only receives asset events produced by Dags within the same team.
+This prevents unintended cross-team triggers.
+
+To allow specific other teams to produce events that trigger your Dag, use the ``allow_teams`` parameter


I find allow_teams a big vague. Maybe allow_producing_teams or allow_teams_to_trigger or something along those lines?

o-nikolas · 2026-04-22T22:47:58Z

+- A consuming Dag only receives events from Dags in the same team.
+- Dags with no team association (global Dags) can produce events that trigger any consumer.


The interplay between global produces and consumers that are team or not team, is kind of vague/confusing in this section. Is there a way we can make that a bit more clear/concrete?

o-nikolas · 2026-04-22T22:52:30Z

+By default, a consuming Dag only receives asset events from producers within the same team. Dags with
+no team association (global Dags) act as global producers and can trigger any team-bound consumer.


Something about the second sentence invalidating the first one I find slightly confusing. I'd maybe make them less exclusive to one another?

Suggested change

By default, a consuming Dag only receives asset events from producers within the same team. Dags with

no team association (global Dags) act as global producers and can trigger any team-bound consumer.

By default, a consuming Dag only receives asset events from producers within the same team or from Dags with

no team association, i.e. global Dags. These global Dags act as global producers and can trigger any team-bound or non-team-bound consumer.

NOTE:
I think the second sentence could be dropped honestly, but kept it because it had some of your original wording

o-nikolas · 2026-04-22T22:53:28Z

+Cross-Team Opt-In with ``allow_teams``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To allow specific teams to produce events that trigger consumers on a given asset, use the


Just a nit, I think this clarifies more.

Suggested change

To allow specific teams to produce events that trigger consumers on a given asset, use the

To allow specific teams to produce events that trigger consumers on a given asset from another team, use the

o-nikolas · 2026-04-22T22:54:15Z

+Behavioral Rules
+^^^^^^^^^^^^^^^^
+
+The following truth table describes the complete filtering logic:


Suggested change

The following truth table describes the complete filtering logic:

The following table describes the complete filtering logic:

"truth table" is an overly technical term. Many readers won't know it (and don't need to).

o-nikolas · 2026-04-22T22:54:56Z

+
+- **Same team**: Always allowed.
+- **Global (teamless) DAG producer**: Triggers all consumers regardless of team.
+- **Teamless API user**: Can only trigger teamless consumers.


Why is a teamless API request treated differently from a teamless Dag?

o-nikolas · 2026-04-22T22:55:36Z

+- **Same team**: Always allowed.
+- **Global (teamless) DAG producer**: Triggers all consumers regardless of team.
+- **Teamless API user**: Can only trigger teamless consumers.
+- **Teamless consumer**: Only accepts events from teamless sources.


Hmm, do we need this restriction? If global dags can be a producer for any team, why do we want to stop any team from being a producer for a global consumer?

o-nikolas · 2026-04-22T22:56:56Z

 .. image:: /img/multi_team_arch_diagram.png
   :alt: Multi-Team Airflow Architecture showing resource isolation between teams

+.. _multi-team-asset-event-filtering:


Can you shuffle this above the Architecture diagram? I think that overall diagram is a nice generic way to conclude the doc and a specifically targeted section should be above? Or were you trying to keep it close to the scheduling section?

vincbeck requested a review from o-nikolas April 22, 2026 18:48

boring-cyborg Bot added the kind:documentation label Apr 22, 2026

Add documentation for team-based asset event filtering

ae93d2c

vincbeck force-pushed the vincbeck/mt_doc branch from 45328a3 to ae93d2c Compare April 22, 2026 18:54

vincbeck requested a review from potiuk April 22, 2026 19:56

o-nikolas reviewed Apr 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add documentation for team-based asset event filtering#65690

Add documentation for team-based asset event filtering#65690
vincbeck wants to merge 1 commit intoapache:mainfrom
aws-mwaa:vincbeck/mt_doc

vincbeck commented Apr 22, 2026

Uh oh!

boring-cyborg Bot commented Apr 22, 2026

Uh oh!

vincbeck commented Apr 22, 2026

Uh oh!

o-nikolas left a comment

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

o-nikolas Apr 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		- A consuming Dag only receives events from Dags in the same team.
		- Dags with no team association (global Dags) can produce events that trigger any consumer.

		By default, a consuming Dag only receives asset events from producers within the same team. Dags with
		no team association (global Dags) act as global producers and can trigger any team-bound consumer.

	To allow specific teams to produce events that trigger consumers on a given asset, use the
	To allow specific teams to produce events that trigger consumers on a given asset from another team, use the

	The following truth table describes the complete filtering logic:
	The following table describes the complete filtering logic:

Conversation

vincbeck commented Apr 22, 2026

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborg Bot commented Apr 22, 2026

Uh oh!

vincbeck commented Apr 22, 2026

Uh oh!

o-nikolas left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants