Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Thanks for considering a contribution. This repo is the worker node for [Greffon
By signing off your commits you certify that you wrote the code (or have the right to submit it) and license it to the project under [AGPL v3](./LICENSE). Full text at [developercertificate.org](https://developercertificate.org/).

```bash
git commit -s -m "feat: add CRL refresh backoff"
git commit -s -m "feat: add monitor status backoff"
```

PRs without DCO sign-off cannot be merged.
Expand All @@ -34,7 +34,7 @@ GREFFER_WORKERS_ENABLED=true GREFFER_PROTOCOL=http \
poetry run uvicorn --factory app.main:create_app --host 0.0.0.0 --port 8001
```

`source env.env` is required — `Settings` has no auto-loaded env file and `apply_ops_migrations` exits early without `GREFFER_ID`. `GREFFER_WORKERS_ENABLED=true` turns on the register/monitor/CRL tasks. `GREFFER_PROTOCOL=http` overrides env.env's `https` default: bare uvicorn has no TLS, so without the override the greffer registers an `https` callback URL that the manager can't reach. (Run the full compose stack if you want production-like nginx TLS.) The greffer also needs a reachable manager and a Docker daemon. See [README.md](./README.md) for the env var set.
`source env.env` is required — `Settings` has no auto-loaded env file and `apply_ops_migrations` exits early without `GREFFER_ID`. `GREFFER_WORKERS_ENABLED=true` turns on the register/monitor tasks. `GREFFER_PROTOCOL=http` overrides env.env's `https` default: bare uvicorn has no TLS, so without the override the greffer registers an `https` callback URL that the manager can't reach. (Run the full compose stack if you want production-like nginx TLS.) The greffer also needs a reachable manager and a Docker daemon. See [README.md](./README.md) for the env var set.

## Code style

Expand All @@ -44,7 +44,7 @@ GREFFER_WORKERS_ENABLED=true GREFFER_PROTOCOL=http \
- Use `transaction`-style atomic patterns for multi-step on-disk state changes
- Never `verify=False` on `requests`/`httpx` calls
- Secrets from environment variables only — never hardcoded
- Respect the `--workers 1` invariant: background tasks (register / monitor / CRL sync) assume a single process
- Respect the `--workers 1` invariant: background tasks (register / monitor) assume a single process

## Tests

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ COPY . /app
# than the pre-cutover `;` which started anyway.
#
# --workers 1 is a hard requirement: multi-worker uvicorn would spawn N
# copies of each background task (register / monitor / CRL sync), each
# copies of each background task (register / monitor), each
# minting its own token and fighting the manager over cert state. See
# HLD #3 § Single-worker uvicorn constraint.
CMD ["sh", "-c", "poetry run python -m app.cli apply_ops_migrations && exec poetry run uvicorn --factory app.main:create_app --host 0.0.0.0 --port 8000 --workers 1"]
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Docker Engine (docker-compose) ──► greffon instances
Nginx (per-instance TLS reverse proxy)
```

The greffer runs as a single uvicorn worker (`--workers 1`) on purpose: three background tasks (register / monitor / CRL sync) live in one process, and multi-worker would spawn duplicate copies that fight over cert state.
The greffer runs as a single uvicorn worker (`--workers 1`) on purpose: two background tasks (register / monitor) live in one process, and multi-worker would spawn duplicate copies that fight over cert state.

## Local development

Expand All @@ -46,7 +46,7 @@ set -a && source env.env && set +a
# Run on-disk state migrations before the server binds (they must not race request handlers)
poetry run python -m app.cli apply_ops_migrations

# GREFFER_WORKERS_ENABLED=true turns on the register / monitor / CRL background tasks —
# GREFFER_WORKERS_ENABLED=true turns on the register / monitor background tasks —
# without it the greffer starts but never registers with the manager or sends callbacks.
# GREFFER_PROTOCOL=http: bare uvicorn serves plain HTTP. env.env defaults to https because
# the full compose stack puts an nginx TLS proxy in front; running uvicorn directly has no
Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Include as much as you can:

## Scope — especially relevant for the greffer

- mTLS certificate / private key handling and the CRL sync path
- TLS certificate / private key handling (cert install path)
- Jinja2 rendering of catalog compose templates (template injection, arbitrary host paths)
- Docker SDK usage (container escape, host mount exposure)
- The Nginx per-instance proxy config generation
Expand Down
2 changes: 1 addition & 1 deletion app/lifespan.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
"""App lifespan — start the three background workers if enabled.
"""App lifespan — start the background workers if enabled.

``greffer_workers_enabled`` defaults to False so unit tests don't
accidentally spawn real workers. Production sets
Expand Down
3 changes: 1 addition & 2 deletions app/settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,12 +89,11 @@ def _empty_string_is_none(cls, v):

docker_nginx_name: str = "greffer-nginx-1"

crl_sync_interval: int = 300
monitor_interval: int = 5

skip_ops_migrations: bool = False

# Workers (register / monitor_status / CRL sync). Disabled by default
# Workers (register / monitor_status). Disabled by default
# so unit tests don't accidentally start real workers. Production
# enables via ``GREFFER_WORKERS_ENABLED=true`` in compose.
#
Expand Down
13 changes: 8 additions & 5 deletions app/workers/__init__.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
"""Background workers — greffer lifecycle tasks running under FastAPI lifespan.

Three workers port the Django daemon threads at ``apps/controller/views.py:14-22``:
Two workers port the Django daemon threads at ``apps/controller/views.py:14-22``:

* ``register_worker`` — one-shot; POSTs registration + polls for cert.
* ``monitor_worker`` — forever loop; reports greffon instance status changes.
* ``crl_sync_worker`` — forever loop; fetches updated CRL periodically.

(A third legacy worker, ``crl_sync_worker``, copied the manager's CRL
into the nginx container. Removed: no nginx config ever loaded the file
(``ssl_crl`` requires client-cert verification, which this nginx does
not do), so the sync had no effect. Revocation enforcement is deferred
to the platform's planned step-ca migration.)

Startup is gated by ``Settings.greffer_workers_enabled`` (env var
``GREFFER_WORKERS_ENABLED``, default False) so unit tests don't
accidentally spawn real workers. Production sets it to ``true`` in
``docker-compose.yml``.

Single-worker uvicorn only. Multi-worker would spawn N × 3 pollers per
Single-worker uvicorn only. Multi-worker would spawn N × 2 pollers per
container, each with a distinct token, fighting over manager state.
"""
from __future__ import annotations
Expand All @@ -21,7 +26,6 @@

from fastapi import FastAPI

from app.workers.crl import crl_sync_worker
from app.workers.monitor import monitor_worker
from app.workers.register import register_worker

Expand All @@ -32,7 +36,6 @@ def start_workers(app: FastAPI) -> list[asyncio.Task]:
return [
asyncio.create_task(register_worker(app), name="greffer-register"),
asyncio.create_task(monitor_worker(app), name="greffer-monitor"),
asyncio.create_task(crl_sync_worker(app), name="greffer-crl-sync"),
]


Expand Down
39 changes: 0 additions & 39 deletions app/workers/crl.py

This file was deleted.

29 changes: 1 addition & 28 deletions app/workers/register.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Register worker — POST to manager, poll for cert, install, fetch initial CRL.
"""Register worker — POST to manager, poll for cert, install.

Ports ``apps/utils/greffon/base_server.py:register()`` to asyncio. The outer
loops are rewritten as async so they cancel cleanly on lifespan shutdown;
Expand Down Expand Up @@ -102,9 +102,6 @@ async def register_worker(app: FastAPI) -> None:
data,
abandon_on_cancel=True,
)
await anyio.to_thread.run_sync(
_fetch_and_store_crl, settings, abandon_on_cancel=True
)
logger.info("register complete")
return
await asyncio.sleep(_CERT_POLL_SECONDS)
Expand Down Expand Up @@ -221,27 +218,3 @@ def _maybe_install_initial_tunnel_config(
return
if wrote:
logger.info("initial_tunnel_config_installed path=%s", target)


def _fetch_and_store_crl(settings: Settings) -> None:
"""Fetch CRL from manager, copy into nginx container. Shared with
``crl_sync_worker`` (see ``app/workers/crl.py``)."""
from apps.utils.docker.base import copy_file_into_container

try:
res = requests.get(
f"{settings.greffon_base_server}/api/greffer/ca/crl/",
verify=settings.greffer_ssl_verify,
timeout=10,
)
if res.status_code == 200:
copy_file_into_container(
settings.docker_nginx_name, "/root", "revoked.crl", res.text
)
logger.info("CRL updated successfully")
else:
logger.warning("Failed to fetch CRL: HTTP %s", res.status_code)
except Exception as e:
# Preserve legacy behavior: log and continue. The caller's async
# loop handles retry timing.
logger.warning("Failed to fetch CRL: %s", e)
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ services:
environment:
# Feature #3 shipped dormant; flip it on here at the cutover moment.
# Without this, the greffer would boot with the HTTP surface alive
# but no register / monitor / CRL workers — the manager would never
# but no register / monitor workers — the manager would never
# see a cert handshake or instance-status callbacks.
- GREFFER_WORKERS_ENABLED=true
extra_hosts:
Expand Down
8 changes: 3 additions & 5 deletions tests/test_lifespan.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ async def test_lifespan_no_tasks_when_workers_disabled(


@pytest.mark.asyncio
async def test_lifespan_starts_three_workers_when_enabled(
async def test_lifespan_starts_workers_when_enabled(
settings: Settings,
) -> None:
"""``greffer_workers_enabled=True`` → three tasks started with expected
"""``greffer_workers_enabled=True`` → both tasks started with expected
names, cancelled on shutdown."""
settings.greffer_workers_enabled = True # type: ignore[misc]
app = create_app(token="t", settings=settings)
Expand All @@ -54,20 +54,18 @@ async def _noop_worker(_app):
# reaches them before `start_workers` looks them up.
with patch("app.workers.register_worker", new=_noop_worker), patch(
"app.workers.monitor_worker", new=_noop_worker
), patch("app.workers.crl_sync_worker", new=_noop_worker):
):
async with lifespan(app):
current_names = {
t.get_name() for t in asyncio.all_tasks() if not t.done()
}
assert "greffer-register" in current_names
assert "greffer-monitor" in current_names
assert "greffer-crl-sync" in current_names

# After lifespan shutdown the worker tasks must be gone.
leftover = {t.get_name() for t in asyncio.all_tasks() if not t.done()}
assert "greffer-register" not in leftover
assert "greffer-monitor" not in leftover
assert "greffer-crl-sync" not in leftover


def test_greffer_workers_enabled_env_var_binds(
Expand Down
5 changes: 2 additions & 3 deletions tests/test_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ def test_defaults_apply_when_env_unset(monkeypatch: pytest.MonkeyPatch) -> None:
assert s.greffer_public_scheme == "https"
assert s.greffon_path == Path("/data")
assert s.docker_nginx_name == "greffer-nginx-1"
assert s.crl_sync_interval == 300
assert s.monitor_interval == 5
assert s.skip_ops_migrations is False
assert s.logger_name == "greffer"
Expand All @@ -41,11 +40,11 @@ def test_defaults_apply_when_env_unset(monkeypatch: pytest.MonkeyPatch) -> None:
def test_env_overrides_defaults(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("GREFFER_ID", "x")
monkeypatch.setenv("GREFFON_PATH", "/tmp/alt")
monkeypatch.setenv("CRL_SYNC_INTERVAL", "60")
monkeypatch.setenv("MONITOR_INTERVAL", "60")
monkeypatch.setenv("GREFFER_SSL_VERIFY", "false")
s = Settings()
assert s.greffon_path == Path("/tmp/alt")
assert s.crl_sync_interval == 60
assert s.monitor_interval == 60
assert s.greffer_ssl_verify is False


Expand Down
86 changes: 0 additions & 86 deletions tests/test_workers_crl.py

This file was deleted.

Loading