Switch from milksnake to a more modern approach for Python-Rust bindings

[juhoinkinen]

Milksnake is not maintained anymore and using it seems problematic at least with pyproject.toml: NatLibFi#7

Alternatives seem to be switch to setuptools-rust or Maturin. Maturin seems more widely used, but on my AI-assisted attempt to switch to Maturin Qwen3 actually pushed back and stated

maturin is primarily designed for PyO3-based projects. Your project uses CFFI for Python bindings (loading a C shared library via ffi.dlopen()). For maturin to work with CFFI, we'd need to load the bundled .so/.dylib at runtime rather than as a CFFI extension module. Alternatively, setuptools-rust would be a more natural fit since it integrates with your existing setuptools + CFFI setup. Would you like me to proceed with maturin + CFFI runtime loading, or would you prefer setuptools-rust as a more seamless modernization?

So maybe it would be better to use setuptools-rust instead.

[juhoinkinen]

@ronja-koistinen do you have any advice for this?

[ronja-koistinen]

I looked at both, but I don't really know enough about Python to be able to judge this. Sorry.

[juhoinkinen]

Copilot recommends Maturin, I'll try that:

Recommendation: Migrate to Maturin (not setuptools-rust) for this project.

Here's the comparison:

Factor	setuptools-rust	Maturin	Winner
Setup complexity	Moderate—requires Cargo.toml in Rust dir, build script	Minimal—single pyproject.toml, batteries-included	Maturin
Platform-specific builds (macOS universal2)	Manual subprocess calls still needed in build script	Built-in support for multiple targets via config	Maturin
Build speed	Slower—re-runs for each platform	Faster—parallel multi-target builds	Maturin
PEP 517 compliance	Partial—still relies on setup.py	Full—modern standards-based	Maturin
Wheel generation	Manual integration	Automatic, optimized wheels	Maturin
CFFI wrapping	No built-in support	Native Python extension (no CFFI needed)	Maturin
Project structure complexity	Keeps current separation (c-api/, python-wrapper/)	Can consolidate into single layout	Maturin
Community/maintenance	Stable but older	Active, modern, industry standard	Maturin

Why Maturin wins for this project:

1. Eliminates manual build logic: Your current code manually handles macOS universal2 builds (lines 15-49). Maturin handles this via configuration—just set targets = ["x86_64-apple-darwin", "aarch64-apple-darwin"] in pyproject.toml.

2. No CFFI overhead: Your code wraps the C library via CFFI (spec.add_cffi_module). Maturin builds a direct Python extension—simpler, faster, no additional runtime.

3. Better wheel compatibility: Maturin generates optimized wheels automatically for PyPI. setuptools-rust still requires manual wheel management.

4. Modern standards: Maturin is the de facto standard for Rust↔Python bindings (used by cryptography, PyO2, Pydantic, etc.). setuptools-rust is less actively maintained.

5. Cleaner repo layout: Maturin works well with your current structure but can also simplify it if desired.

Migration path:

• Create pyproject.toml with Maturin configuration (specify targets, entry points)
• Keep setup.py minimal or remove it (Maturin handles PEP 517)
• Delete the build_native() function and milksnake code
• Update c-api/Cargo.toml to use pyo3 crate (Maturin's native Python binding framework)

setuptools-rust would work, but it's essentially a middle ground—you'd still need build scripts for the macOS universal2 logic, and you lose Maturin's modern ergonomics.