Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/basic-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -352,8 +352,8 @@ jobs:
--answer yes --dry-run
- name: "Test providers metadata generation"
run: |
git remote add apache https://github.com/apache/airflow.git
git fetch apache --tags
git remote add upstream https://github.com/apache/airflow.git
git fetch upstream --tags
breeze release-management generate-providers-metadata --refresh-constraints-and-airflow-releases
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
79 changes: 68 additions & 11 deletions AGENTS.md
Comment thread
potiuk marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -150,18 +150,75 @@ Add a newsfragment for user-visible changes:

- NEVER add Co-Authored-By with yourself as co-author of the commit. Agents cannot be authors, humans can be, Agents are assistants.

### Git remote naming conventions

Airflow standardises on two git remote names, and the rest of this file, the
contributing docs, and the release docs all assume them:

- **`upstream`** — the canonical `apache/airflow` repository (fetch from here).
- **`origin`** — the contributor's fork of `apache/airflow` (push PR branches here).

Always push branches to `origin`. Never push directly to `upstream` (and never
push directly to `main` on either remote).

**Before running any remote-based command, run `git remote -v` and verify the
names match this convention.** If they do not — for example, the upstream remote
is called `apache`, or `origin` points at `apache/airflow` with the fork under a
different name like `fork` — **do not silently go along with the existing
names**. Surface the mismatch to the user and propose the exact rename commands
to bring the checkout in line with the convention, then ask the user to confirm
before running them. Examples:

- Upstream is named `apache`, fork is `origin` (common legacy layout):

```bash
git remote rename apache upstream
```

- `origin` points at `apache/airflow` and the fork is named `fork` (release-manager
/ "cloned upstream directly" layout):

```bash
git remote rename origin upstream
git remote rename fork origin
```

- Upstream is missing entirely:

```bash
git remote add upstream https://github.com/apache/airflow.git
# or, for SSH:
git remote add upstream git@github.com:apache/airflow.git
```

- Fork is missing entirely:

```bash
gh repo fork apache/airflow --remote --remote-name origin
```

After any rename/add, re-run `git remote -v` to confirm the new state before
continuing with commands that assume `upstream` / `origin`.

If a doc, script, or command you're about to run uses the old `apache` name (or
any other variant), **translate it to the `upstream` convention** in what you
propose to the user, rather than perpetuating the old name. Flag the stale
documentation so it can be fixed in a follow-up.

### Creating Pull Requests

**Always push to the user's fork**, not to the upstream `apache/airflow` repo. Never push
directly to `main`.
**Always push to the user's fork (`origin`)**, not to `upstream` (`apache/airflow`).
Never push directly to `main`.

Before pushing, confirm the remote setup matches the conventions above
(`upstream` → `apache/airflow`, `origin` → your fork). Run `git remote -v` and,
if the names don't match, propose renames as described in "Git remote naming
conventions" — ask the user to confirm before running them.

Before pushing, determine the fork remote. Check `git remote -v` — if `origin` does **not**
point to `apache/airflow`, use `origin` (it's the user's fork). If `origin` points to
`apache/airflow`, look for another remote that points to the user's fork. If no fork remote
exists, create one:
If the fork remote does not exist at all, create one:

```bash
gh repo fork apache/airflow --remote --remote-name fork
gh repo fork apache/airflow --remote --remote-name origin
```

Before pushing, perform a self-review of your changes following the Gen-AI review guidelines
Expand All @@ -186,18 +243,18 @@ Before pushing, always rebase your branch onto the latest target branch (usually
to avoid merge conflicts and ensure CI runs against up-to-date code:

```bash
git fetch <upstream-remote> <target_branch>
git rebase <upstream-remote>/<target_branch>
git fetch upstream <target_branch>
git rebase upstream/<target_branch>
```

If there are conflicts, resolve them and continue the rebase. If the rebase is too complex,
ask the user for guidance.

Then push the branch to the fork remote and open the PR creation page in the browser
Then push the branch to your fork (`origin`) and open the PR creation page in the browser
with the body pre-filled (including the generative AI disclosure already checked):

```bash
git push -u <fork-remote> <branch-name>
git push -u origin <branch-name>
gh pr create --web --title "Short title (under 70 chars)" --body "$(cat <<'EOF'
Brief description of the changes.

Expand Down
66 changes: 52 additions & 14 deletions contributing-docs/10_working_with_git.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,44 @@ that we are using rebase workflow. It also explains how to sync your fork with t
:depth: 2
:local:

Git remote naming conventions
=============================

Airflow documentation, helper scripts (``dev/sync_fork.sh``), release tooling and
agent instructions (``AGENTS.md``) all assume the following two remote names, and
you should configure your local checkout to match:

* ``upstream`` — the canonical ``apache/airflow`` repository (where you fetch from).
* ``origin`` — your personal fork of ``apache/airflow`` (where you push branches
for PRs).

Always push PR branches to ``origin``; don't push to ``upstream`` (the branch
protection on ``apache/airflow`` will reject it anyway). Working on dedicated
branches is recommended, though developing directly on your fork's ``main`` is
tolerated — see `Contribution Workflow </contributing-docs/18_contribution_workflow.rst#step-4-prepare-pr>`_.

If your existing checkout uses different names (for example ``apache`` for the
Apache remote, or ``origin`` pointing at ``apache/airflow`` with your fork under
another name), rename them to match the convention. Common migrations:

.. code-block:: bash

# Case 1: upstream is currently named "apache"
git remote rename apache upstream

# Case 2: "origin" points at apache/airflow and your fork is named "fork"
git remote rename origin upstream
git remote rename fork origin

# Case 3: upstream is missing entirely
git remote add upstream https://github.com/apache/airflow.git
# ... or via SSH:
git remote add upstream git@github.com:apache/airflow.git

Then confirm with ``git remote -v``. Ad-hoc remote names still work for one-off
commands, but the helper scripts and documented workflows below all assume
``upstream`` / ``origin``.

Airflow Git Branches
====================

Expand Down Expand Up @@ -68,7 +106,7 @@ How to sync your fork

When you have your fork, you should periodically synchronize the main of your fork with the
Apache Airflow main. In order to do that you can ``git pull --rebase`` to your local git repository from
apache remote and push the main (often with ``--force`` to your fork). There is also an easy
the ``upstream`` remote and push the main (often with ``--force`` to your fork). There is also an easy
way to sync your fork in GitHub's web UI with the `Fetch upstream feature
<https://docs.github.com/en/github/collaborating-with-pull-requests/working-with-forks/syncing-a-fork#syncing-a-fork-from-the-web-ui>`_.

Expand Down Expand Up @@ -139,30 +177,30 @@ First of all, we suggest you read about the rebase workflow here:
`Merging vs. rebasing <https://www.atlassian.com/git/tutorials/merging-vs-rebasing>`_. This is an
excellent article that describes all the ins/outs of the rebase workflow. I recommend keeping it for future reference.

The goal of rebasing your PR on top of ``apache/main`` is to "transplant" your change on top of
The goal of rebasing your PR on top of ``upstream/main`` is to "transplant" your change on top of
the latest changes that are merged by others. It also allows you to fix all the conflicts
that arise as a result of other people changing the same files as you and merging the changes to ``apache/main``.
that arise as a result of other people changing the same files as you and merging the changes to ``upstream/main``.

Here is how rebase looks in practice (you can find a summary below these detailed steps):

1. You first need to add the Apache project remote to your git repository. This is only necessary once,
so if it's not the first time you are following this tutorial you can skip this step. In this example,
we will be adding the remote as "apache" so you can refer to it easily
so if it's not the first time you are following this tutorial you can skip this step. Per the
`Git remote naming conventions`_ we add it as ``upstream``:

* If you use ssh: ``git remote add apache git@github.com:apache/airflow.git``
* If you use https: ``git remote add apache https://github.com/apache/airflow.git``
* If you use ssh: ``git remote add upstream git@github.com:apache/airflow.git``
* If you use https: ``git remote add upstream https://github.com/apache/airflow.git``
Copy link

Copilot AI Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the numbered rebase steps, the continuation lines and the nested bullet list under step 1 are not indented, which breaks the reStructuredText list structure (the bullets/paragraphs will render outside the "1." item). Indent the wrapped paragraph lines and the * If you use ... bullets to be part of step 1 (typically 3 spaces past the list marker) so Sphinx renders the list correctly.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indented the wrapped paragraph and both bullets under step 1 to 3 spaces so the list renders correctly in Sphinx. Fixed in 475f8ff.


2. You then need to make sure that you have the latest main fetched from the ``apache`` repository. You can do this
2. You then need to make sure that you have the latest main fetched from the ``upstream`` repository. You can do this
via

``git fetch apache`` (to fetch apache remote)
``git fetch upstream`` (to fetch upstream remote)

``git fetch --all`` (to fetch all remotes)

3. Assuming that your feature is in a branch in your repository called ``my-branch`` you can easily check
what is the base commit you should rebase from via

``git merge-base my-branch apache/main``
``git merge-base my-branch upstream/main``

This will print the HASH of the base commit which you should use to rebase your feature from.
For example: ``5abce471e0690c6b8d06ca25685b0845c5fd270f``. Copy that HASH and go to the next step.
Expand Down Expand Up @@ -201,11 +239,11 @@ we will be adding the remote as "apache" so you can refer to it easily

6. Rebase

``git rebase HASH --onto apache/main``
``git rebase HASH --onto upstream/main``

For example:

``git rebase 5abce471e0690c6b8d06ca25685b0845c5fd270f --onto apache/main``
``git rebase 5abce471e0690c6b8d06ca25685b0845c5fd270f --onto upstream/main``

Rebasing is a good practice recommended to follow for all code changes.

Expand Down Expand Up @@ -249,9 +287,9 @@ Useful when you understand the flow but don't remember the steps and want a quic
git fetch --all
git add .
git commit
git merge-base my-branch apache/main
git merge-base my-branch upstream/main
git checkout my-branch
git rebase HASH --onto apache/main
git rebase HASH --onto upstream/main
git push --force-with-lease

-------
Expand Down
15 changes: 9 additions & 6 deletions contributing-docs/18_contribution_workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -165,15 +165,18 @@ Step 4: Prepare PR
`How to sync your fork <10_working_with_git.rst#how-to-sync-your-fork>`_ for details.

* Create a local branch for your development. Make sure to use latest
``apache/main`` as base for the branch. See `How to Rebase PR </contributing-docs/10_working_with_git.rst#how-to-rebase-pr>`_ for some details
on setting up the ``apache`` remote. Note, some people develop their changes directly in their own
``main`` branches - this is OK and you can make PR from your main to ``apache/main`` but we
``upstream/main`` as base for the branch. See `How to Rebase PR </contributing-docs/10_working_with_git.rst#how-to-rebase-pr>`_ for details
on setting up the ``upstream`` remote (Airflow standardises on ``upstream`` →
``apache/airflow`` and ``origin`` → your fork — see
`Git remote naming conventions </contributing-docs/10_working_with_git.rst#git-remote-naming-conventions>`_).
Note, some people develop their changes directly in their own
``main`` branches - this is OK and you can make PR from your main to ``upstream/main`` but we
Comment thread
potiuk marked this conversation as resolved.
recommend to always create a local branch for your development. This allows you to easily compare
changes, have several changes that you work on at the same time and many more.
If you have ``apache`` set as remote then you can make sure that you have latest changes in your main
by ``git pull apache main`` when you are in the local ``main`` branch. If you have conflicts and
With ``upstream`` configured you can make sure that you have the latest changes in your main
by ``git pull upstream main`` when you are in the local ``main`` branch. If you have conflicts and
want to override your locally changed main you can override your local changes with
``git fetch apache; git reset --hard apache/main``.
``git fetch upstream; git reset --hard upstream/main``.

* Modify the class and add necessary code and unit tests.

Expand Down
8 changes: 5 additions & 3 deletions dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,13 +279,15 @@ This is a step-by-step instruction on how to use it:
1. You need to have "airflow" repository checked out separately from the repository you are working on. For
example in `/home/myuser/airflow-constraints` folder.
2. You need to checkout `constraints-main` branch in this repository. By default the command expects that
there is a remote named "apache" pointing to the official Apache repository. You can override this
by passing `--remote-name` option to the command.
there is a remote named "upstream" pointing to the `apache/airflow` repository (the standard remote
naming convention — see
[`contributing-docs/10_working_with_git.rst`](../contributing-docs/10_working_with_git.rst#git-remote-naming-conventions)).
You can override this by passing `--remote-name` option to the command.
3. You need to run `breeze release-management update-constraints` command. The `breeze` command comes usually
Copy link
Copy Markdown
Collaborator

@Dev-iL Dev-iL Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider rewriting this bullet to avoid using "usually" twice. Suggestions by gemini:

  • ​Option 1 (Direct):
    You need to run the breeze release-management update-constraints command. Typically, breeze is run from a separate clone of the Airflow repository, specifically from the main branch.
    ​- Option 2 (Concise):
    Run the breeze release-management update-constraints command. This usually stems from another clone of the Airflow repository (often the main branch).
  • ​Option 3 (Action-oriented):
    You need to run breeze release-management update-constraints. Note that the breeze command generally comes from a different Airflow clone, specifically the one tracking the main branch.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied option 1 ("Typically, breeze is run from a separate clone of the Airflow repository, on the main branch.") in 475f8ff — thanks!

from another clone of airflow repository - usually from the `main` branch. You should pass those options to
the command:
* path to the "constraints" repository
* remote name where the constraints should be pushed (optionally - default "apache")
* remote name where the constraints should be pushed (optionally - default "upstream")
* list of airflow versions to update constraints for
* list of constraints to update in the form of "package==version" (you can specify it multiple times)
* message to be used in the commit message
Expand Down
41 changes: 28 additions & 13 deletions dev/README_RELEASE_AIRFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,18 +297,32 @@ You are likely want to cherry-pick some of the latest doc changes in order to br
explanations added to the documentation. Usually you can see the list of such changes via:

```shell
git fetch apache
git log --oneline apache/v3-2-test | sed -n 's/.*\((#[0-9]*)\)$/\1/p' > /tmp/merged
git log --oneline --decorate apache/v2-2-stable..apache/main -- docs/apache-airflow docs/docker-stack/ | grep -vf /tmp/merged
git fetch upstream
git log --oneline upstream/v3-2-test | sed -n 's/.*\((#[0-9]*)\)$/\1/p' > /tmp/merged
git log --oneline --decorate upstream/v2-2-stable..upstream/main -- docs/apache-airflow docs/docker-stack/ | grep -vf /tmp/merged
```

Those changes that are "doc-only" changes should be marked with `type:doc-only` label so that they
land in documentation part of the changelog. The tool to review and assign the labels is described below.

## Making the cherry picking

It is recommended to clone Airflow upstream (not your fork) and run the commands on
the relevant test branch in this clone. That way origin points to the upstream repo.
It is recommended to clone Airflow from `apache/airflow` directly (not your fork) into a
dedicated release-manager checkout and run the commands on the relevant test branch there.
This repo follows the standard convention that `upstream` → `apache/airflow` and `origin`
→ your fork (see
[`contributing-docs/10_working_with_git.rst`](../contributing-docs/10_working_with_git.rst#git-remote-naming-conventions)),
so in this release-manager clone add `apache/airflow` as `upstream`:

```shell
git remote add upstream https://github.com/apache/airflow.git
git fetch upstream
```

All the commands in this document assume `upstream` is the remote that tracks
`apache/airflow`. If you previously set this up under a different name (e.g. `apache`),
either rename it with `git remote rename apache upstream` or pass the alternative name
via the `--remote-name` option where the commands accept it.

To see cherry picking candidates (unmerged PR with the appropriate milestone), from the test
branch you can run:
Expand Down Expand Up @@ -344,7 +358,7 @@ We have the tool that allows to review cherry-picked PRs and assign the labels
It allows to manually review and assign milestones and labels to cherry-picked PRs:

```shell
./dev/assign_cherry_picked_prs_with_milestone.py assign-prs --previous-release v2-2-stable --current-release apache/v2-2-test --milestone-number 48
./dev/assign_cherry_picked_prs_with_milestone.py assign-prs --previous-release v2-2-stable --current-release upstream/v2-2-test --milestone-number 48
```

It summarises the state of each cherry-picked PR including information whether it is going to be
Expand All @@ -360,7 +374,7 @@ assumes the `--skip-assigned` flag, so that the summary can be produced without

```shell
./dev/assign_cherry_picked_prs_with_milestone.py assign-prs --previous-release v2-2-stable \
--current-release apache/v2-2-test --milestone-number 48 --skip-assigned --assume-yes --print-summary \
--current-release upstream/v2-2-test --milestone-number 48 --skip-assigned --assume-yes --print-summary \
--output-folder /tmp
```

Expand All @@ -386,13 +400,13 @@ git show --format=tformat:"" --stat --name-only $(cat /tmp/doc-only-changes.txt)
Then if you see suspicious file (example airflow/sensors/base.py) you can find details on where they came from:

```shell
git log apache/v3-2-test --format="%H" -- airflow/sensors/base.py | grep -f /tmp/doc-only-changes.txt | xargs git show
git log upstream/v3-2-test --format="%H" -- airflow/sensors/base.py | grep -f /tmp/doc-only-changes.txt | xargs git show
```

And the URL to the PR it comes from:

```shell
git log apache/v3-2-test --format="%H" -- airflow/sensors/base.py | grep -f /tmp/doc-only-changes.txt | \
git log upstream/v3-2-test --format="%H" -- airflow/sensors/base.py | grep -f /tmp/doc-only-changes.txt | \
xargs -n 1 git log --oneline --max-count=1 | \
sed s'/.*(#\([0-9]*\))$/https:\/\/github.com\/apache\/airflow\/pull\/\1/'
```
Expand Down Expand Up @@ -484,7 +498,7 @@ uv tool install -e ./dev/breeze

```shell script
git checkout v${VERSION_BRANCH}-test
git reset --hard origin/v${VERSION_BRANCH}-test
git reset --hard upstream/v${VERSION_BRANCH}-test
```

- Create a new branch from v${VERSION_BRANCH}-test
Expand Down Expand Up @@ -635,13 +649,14 @@ uv tool install -e ./dev/breeze
Before running the actual release command, you can safely test it using:

```shell script
# Test with dry-run (shows what would be executed without doing it)
# Test with dry-run (shows what would be executed without doing it).
# --remote-name defaults to "upstream" per the project convention, so only
# pass it explicitly if you set apache/airflow up under a different name.
breeze release-management start-rc-process \
--version ${VERSION_RC} \
--previous-version ${PREVIOUS_VERSION} \
--task-sdk-version ${TASK_SDK_VERSION_RC} \
--sync-branch ${SYNC_BRANCH} \
--remote-name upstream \
--dry-run
```

Expand Down Expand Up @@ -899,7 +914,7 @@ VERSION_SUFFIX=rc1
VERSION_RC=${VERSION}${VERSION_SUFFIX}
TASK_SDK_VERSION=X.Y.Z
TASK_SDK_VERSION_RC=${TASK_SDK_VERSION}${VERSION_SUFFIX}
git fetch apache --tags
git fetch upstream --tags
git checkout ${VERSION_RC}
export AIRFLOW_REPO_ROOT=$(pwd)
rm -rf dist/*
Expand Down
Loading
Loading