Use ogr2ogr instead of geopandas - #46

Open

f-necas wants to merge 33 commits into

mainfrom

use-gdal

Commits on Sep 7, 2026

feat(ingestion): add ogr2ogr support for ingesting geospatial files into PostGIS

f-necas
authored and
tonio
committed
3deb5b9
feat(transformation): implement SQL-native transformation pipeline for PostGIS ingestion

f-necas
authored and
tonio
committed
a85c913
feat; bump python to 3.13, airflow to 3.2.2 and add a base image with debian trixie and gdal

f-necas
authored and
tonio
committed
216bbc2
feat: add CI

f-necas
authored and
tonio
committed
6205925
feat: update CI name

f-necas
authored and
tonio
committed
cfca327
feat: add gdal to local_executor.py and use a container to execute it

f-necas
authored and
tonio
committed
9089811
fix(elt): use get_records_as_dicts in process-dag-generator
tonio
committed
8979e9a
fix(ingestion): stop logging ogr2ogr command with credentials
tonio
committed
daf1187
fix(ingestion): remove leftover debug print of ingestion timing
tonio
committed
3228c4e
fix(makefile): build the Airflow base image before `make up`
tonio
committed
43c17c9
perf(ingestion): stream HTTP downloads to disk instead of buffering
tonio
committed
7ed1d4b
fix(ingestion): open ZIP archives through GDAL's /vsizip/ filesystem
tonio
committed
2fee711
fix(ingestion): only assign EPSG:4326 for OGC API - Features
tonio
committed
4975465
fix(transformation): recreate the spatial index after CREATE TABLE AS
tonio
committed
26cb240
chore: drop the chardet dependency orphaned by the ogr2ogr migration
tonio
committed
236911a
fix(ingestion): surface ogr2ogr errors instead of masking them
tonio
committed
8c84307
fix(ingestion): restore shapefile encoding detection for files without .cpg
tonio
committed
9eeacc0
chore(ingestion): remove code left dead by the ogr2ogr migration
tonio
committed
9cc962b
chore(elt): refresh uv.lock after the airflow provider bump
tonio
committed
5cb215a
fix(tests): realign data_manipulation tests with the current code
tonio
committed
c98ae94
fix(docker): give the GDAL sidecar the Parquet driver
tonio
committed
7c3719e
docs(docker): state the real reason the Airflow image needs Trixie
tonio
committed
ec7c1bc
build(docker): base the Airflow image on the GDAL image
tonio
committed
640beaf
fix(elt): correct two pre-existing defects in the DAG params and layer bbox
tonio
committed
f89127a
docs(contribute): fix the local executor make targets and sidecar image
tonio
committed
1246613
ci: run the data_manipulation test suite
tonio
committed
b0c02e2
chore(elt): keep main's Airflow 3.3.0 bump and drop the removed pandas helpers
tonio
committed
49e2135
fix(ingestion): quote the GDAL PG: connection string values
tonio
committed
93a2f35
fix(frontend): unblock npm ci past geonetwork-ui's wrong peer range
tonio
committed
bbeeea3
Merge branch 'fix/frontend-ngx-translate-peers' into use-gdal
tonio
committed
4c12ecc
fix(frontend): regenerate the lockfile with the npm the CI uses
tonio
committed
ac6e1d3
fix(frontend): replace ngx-translate-testing with a local test module
tonio
committed
e412bb9

Commits on Sep 8, 2026

fix(ingestion): accept shapefile archives zipped from the macOS Finder
tonio
committed
95aff37

Navigation Menu

Uh oh!

Use ogr2ogr instead of geopandas - #46

Use ogr2ogr instead of geopandas#46
f-necas wants to merge 33 commits into
mainfrom
use-gdal

Commits on Sep 7, 2026

feat(ingestion): add ogr2ogr support for ingesting geospatial files into PostGIS

feat(transformation): implement SQL-native transformation pipeline for PostGIS ingestion

feat; bump python to 3.13, airflow to 3.2.2 and add a base image with debian trixie and gdal

feat: add CI

feat: update CI name

feat: add gdal to local_executor.py and use a container to execute it

fix(elt): use get_records_as_dicts in process-dag-generator

fix(ingestion): stop logging ogr2ogr command with credentials

fix(ingestion): remove leftover debug print of ingestion timing

fix(makefile): build the Airflow base image before `make up`

perf(ingestion): stream HTTP downloads to disk instead of buffering

fix(ingestion): open ZIP archives through GDAL's /vsizip/ filesystem

fix(ingestion): only assign EPSG:4326 for OGC API - Features

fix(transformation): recreate the spatial index after CREATE TABLE AS

chore: drop the chardet dependency orphaned by the ogr2ogr migration

fix(ingestion): surface ogr2ogr errors instead of masking them

fix(ingestion): restore shapefile encoding detection for files without .cpg

chore(ingestion): remove code left dead by the ogr2ogr migration

chore(elt): refresh uv.lock after the airflow provider bump

fix(tests): realign data_manipulation tests with the current code

fix(docker): give the GDAL sidecar the Parquet driver

docs(docker): state the real reason the Airflow image needs Trixie

build(docker): base the Airflow image on the GDAL image

fix(elt): correct two pre-existing defects in the DAG params and layer bbox

docs(contribute): fix the local executor make targets and sidecar image

ci: run the data_manipulation test suite

chore(elt): keep main's Airflow 3.3.0 bump and drop the removed pandas helpers

fix(ingestion): quote the GDAL PG: connection string values

fix(frontend): unblock npm ci past geonetwork-ui's wrong peer range

Merge branch 'fix/frontend-ngx-translate-peers' into use-gdal

fix(frontend): regenerate the lockfile with the npm the CI uses

fix(frontend): replace ngx-translate-testing with a local test module

Commits on Sep 8, 2026

fix(ingestion): accept shapefile archives zipped from the macOS Finder