Skip to content

Test custom job results #512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
EmileSonneveld wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Test custom job results #512

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
30e3269
Add test_get_job_results_100_server_test
EmileSonneveld Jun 11, 2026
a9063cd
test_get_job_results_100_server_test
EmileSonneveld Jun 11, 2026
009d66f
test_get_job_results_100_from_cwl test_get_job_results_110_from_cwl
EmileSonneveld Jun 11, 2026
9bdd0a3
Fix for job_id
EmileSonneveld Jun 11, 2026
fdcd45a
j-job_metadata_from_cwl
EmileSonneveld Jun 11, 2026
b6da14b
Fix for if collection links are used. Added example_stac_catalog_11 s…
EmileSonneveld Jun 12, 2026
e7fc9a7
Cleaner names in stac examle. Checked with `stac-check example_stac_c…
EmileSonneveld Jun 15, 2026
6e93707
Respond to review
EmileSonneveld Jun 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions openeo_driver/util/stac_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,11 @@ def recurse(item_path: Union[str, Path]) -> list:
href = robust_urljoin(item_path, href)

if "rel" in link and (link["rel"] == "child" or link["rel"] == "item"):
all_files.extend(get_files_from_stac_catalog(href, include_metadata))
else:
all_files.extend(recurse(href))
elif ("rel" not in link) or (link["rel"] != "collection"):
Comment thread
EmileSonneveld marked this conversation as resolved.
Outdated
all_files.append(href)
else:
pass
return all_files

return_files = recurse(catalog_path)
Expand Down
2 changes: 1 addition & 1 deletion openeo_driver/views.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1850,7 +1850,7 @@ def _asset_object(
if filename.endswith(".model"):
# Machine learning models.
return result_dict
bands = asset_metadata.get("bands")
bands = asset_metadata.get("bands") or asset_metadata.get("eo:bands")

@EmileSonneveld EmileSonneveld Jun 12, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soxofaan An issue for CWL here was that the results can be stac1.0, while the code assumed the was 1.1.
Would wrapping this or with TREAT_JOB_RESULTS_V100_LIKE_V110 make more sense?

@EmileSonneveld EmileSonneveld Jun 12, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example with

        bands = asset_metadata.get("bands")
        TREAT_JOB_RESULTS_V100_LIKE_V110 = smart_bool(os.environ.get("TREAT_JOB_RESULTS_V100_LIKE_V110", "0"))
        if not bands and stac11 and TREAT_JOB_RESULTS_V100_LIKE_V110:
            bands = asset_metadata.get("eo:bands")

@soxofaan soxofaan Jun 15, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as far as I understand, TREAT_JOB_RESULTS_V100_LIKE_V110 is a temporary toggle/hack, which I would avoid here

I would just use bands = asset_metadata.get("bands") or asset_metadata.get("eo:bands")

There is a possible problem however that "bands" and "eo:bands" have subtly different sub-structure: e.g. in "bands" you could find "center_wavelength", while in "eo:bands" you would just have "center_wavelength" (without prefix). But I guess the most (only?) important field is just "name", which is the same in both.

@EmileSonneveld EmileSonneveld Jun 15, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the conditional, and now only keeps name property
bands = [{"name": b["name"]} for b in eo_bands]


if bands:
# TODO: #298 this is a quick stop-gap solution for lack of clear API
Expand Down
111 changes: 111 additions & 0 deletions ...ta/example_stac_catalog_11/1b442790-2231-4dd1-b9f4-d3550a512cd1_2023-06-06T00:00:00Z.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
{
"assets": {
"1b442790-2231-4dd1-b9f4-d3550a512cd1_2023-06-06T00:00:00Z_openEO": {
"bands": [
{
"name": "B04",
"statistics": {
"maximum": 1774.0,
"mean": 380.50094473311,
"minimum": 171.0,
"stddev": 176.19790726983,
"valid_percent": 85.92
}
},
{
"name": "B03",
"statistics": {
"maximum": 1470.0,
"mean": 592.27586206896,
"minimum": 282.0,
"stddev": 138.20284171799,
"valid_percent": 85.92
}
},
{
"name": "B02",
"statistics": {
"maximum": 1190.0,
"mean": 412.22508266415,
"minimum": 270.0,
"stddev": 103.22080164099,
"valid_percent": 85.92
}
}
],
"href": "./openEO_2023-06-06Z.tif",
"proj:bbox": [
644530.0,
5675740.0,
645300.0,
5676380.0
],
"proj:code": "EPSG:32631",
"proj:epsg": 32631,
"proj:shape": [
64,
77
],
"roles": [
"data"
],
"title": "s3://openeo-data-dev-waw4-1/batch_jobs/j-2606111953054049bc2368898439491e/openEO_2023-06-06Z.tif",
"type": "image/tiff; application=geotiff"
}
},
"bbox": [
5.069386757303357,
51.214543255519786,
5.080663210429039,
51.22048989375812
],
"collection": "j-2606111953054049bc2368898439491e",
"epsg": 32631,
"geometry": {
"coordinates": [
[
[
5.0804038775948115,
51.214543255519786
],
[
5.080663210429039,
51.22029444279013
],
[
5.069644719194295,
51.22048989375812
],
[
5.069386757303357,
51.21473866656907
],
[
5.0804038775948115,
51.214543255519786
]
]
],
"type": "Polygon"
},
"id": "1b442790-2231-4dd1-b9f4-d3550a512cd1_2023-06-06T00:00:00Z",
"links": [
{
"href": "./collection.json",
"rel": "collection",
"type": "application/json"
}
],
"properties": {
"datetime": "2023-06-06T00:00:00Z",
"proj:code": "EPSG:32631",
"proj:epsg": 32631
},
"stac_extensions": [
"https://stac-extensions.github.io/eo/v1.1.0/schema.json",
"https://stac-extensions.github.io/file/v2.1.0/schema.json",
"https://stac-extensions.github.io/projection/v1.2.0/schema.json"
],
"stac_version": "1.1.0",
"type": "Feature"
}
111 changes: 111 additions & 0 deletions ...ta/example_stac_catalog_11/75625346-7e28-433f-9918-8318369ff7ad_2023-06-01T00:00:00Z.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
{
"assets": {
"75625346-7e28-433f-9918-8318369ff7ad_2023-06-01T00:00:00Z_openEO": {
"bands": [
{
"name": "B04",
"statistics": {
"maximum": 1766.0,
"mean": 401.07085498347,
"minimum": 180.0,
"stddev": 162.43588375276,
"valid_percent": 85.92
}
},
{
"name": "B03",
"statistics": {
"maximum": 1336.0,
"mean": 598.11572980633,
"minimum": 278.0,
"stddev": 133.74416960473,
"valid_percent": 85.92
}
},
{
"name": "B02",
"statistics": {
"maximum": 1072.0,
"mean": 368.17264997638,
"minimum": 216.0,
"stddev": 99.2670146459,
"valid_percent": 85.92
}
}
],
"href": "./openEO_2023-06-01Z.tif",
"proj:bbox": [
644530.0,
5675740.0,
645300.0,
5676380.0
],
"proj:code": "EPSG:32631",
"proj:epsg": 32631,
"proj:shape": [
64,
77
],
"roles": [
"data"
],
"title": "s3://openeo-data-dev-waw4-1/batch_jobs/j-2606111953054049bc2368898439491e/openEO_2023-06-01Z.tif",
"type": "image/tiff; application=geotiff"
}
},
"bbox": [
5.069386757303357,
51.214543255519786,
5.080663210429039,
51.22048989375812
],
"collection": "j-2606111953054049bc2368898439491e",
"epsg": 32631,
"geometry": {
"coordinates": [
[
[
5.0804038775948115,
51.214543255519786
],
[
5.080663210429039,
51.22029444279013
],
[
5.069644719194295,
51.22048989375812
],
[
5.069386757303357,
51.21473866656907
],
[
5.0804038775948115,
51.214543255519786
]
]
],
"type": "Polygon"
},
"id": "75625346-7e28-433f-9918-8318369ff7ad_2023-06-01T00:00:00Z",
"links": [
{
"href": "./collection.json",
"rel": "collection",
"type": "application/json"
}
],
"properties": {
"datetime": "2023-06-01T00:00:00Z",
"proj:code": "EPSG:32631",
"proj:epsg": 32631
},
"stac_extensions": [
"https://stac-extensions.github.io/eo/v1.1.0/schema.json",
"https://stac-extensions.github.io/file/v2.1.0/schema.json",
"https://stac-extensions.github.io/projection/v1.2.0/schema.json"
],
"stac_version": "1.1.0",
"type": "Feature"
}
Loading