Skip to content

Patches for 376 #388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
siamakn merged 2 commits into from
Oct 1, 2026
Merged

Patches for 376 #388

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
393ae01
add waiting helper function
Oct 1, 2026
6fa4317
anonymize author ids
Oct 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 50 additions & 17 deletions docs/tutorial/upload_publish_api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,9 +302,26 @@ This snippet creates a new project for the DFT ZIP file and prints a direct GUI

To check whether entries were created, retrieve them, and print their IDs and URLs, you can type the following:

Processing happens in the background. Define this helper to wait up to ten minutes for it to finish; you can reuse it for the XPS upload and publication below.

```python
import time

def wait_for_process(upload_id, url='test', timeout_in_sec=600):
deadline = time.monotonic() + timeout_in_sec
while time.monotonic() < deadline:
upload = get_upload_by_id(upload_id=upload_id, url=url)
if not upload.process_running:
return upload
time.sleep(5)
raise TimeoutError(f'NOMAD did not finish processing within {timeout_in_sec} seconds.')
```

```python
from nomad_utility_workflows.utils.entries import get_entries_of_upload

wait_for_process(upload_id=dft_upload_id, url='test')

dft_entries = get_entries_of_upload(
upload_id=dft_upload_id, url='test', with_authentication=True
)
Expand All @@ -324,7 +341,7 @@ This snippet retrieves all the entries (here only one entry) created from the up
```

!!! warning
If NOMAD is still processing the project, the list may be empty, or the call may fail with `KeyError: 'entry_metadata'`. Wait until processing has finished and run the cell again. `get_entries_of_upload` keeps its result for up to three minutes, so an empty list may be repeated until then.
If `wait_for_process` times out, inspect the project in the GUI before retrying. Do not query entries while processing is ongoing because an empty result may be cached for up to three minutes.

<!-- TODO: `get_entries_of_upload` raises KeyError: 'entry_metadata' for unprocessed entries and caches its result for 180 s. Simplify this warning (and the 3-minute hint in "Edit the project's metadata") once the package handles both. -->

Expand All @@ -347,7 +364,6 @@ The steps are similar to those you followed for the computations data.
Here is a ready-to-paste snippet for your Jupyter notebook:
```python
import os
import time
from nomad_utility_workflows.utils.uploads import (
upload_files_to_nomad,
get_upload_by_id,
Expand All @@ -360,9 +376,7 @@ The steps are similar to those you followed for the computations data.
xps_upload = get_upload_by_id(xps_upload_id, url='test')
print('Upload GUI URL:', f'{nomad_gui}/projects/{xps_upload.upload_id}')

# wait until NOMAD has finished processing the project
while get_upload_by_id(xps_upload_id, url='test').process_running:
time.sleep(5)
wait_for_process(upload_id=xps_upload_id, url='test')

xps_entries = get_entries_of_upload(
upload_id=xps_upload_id, url='test', with_authentication=True
Expand Down Expand Up @@ -503,14 +517,14 @@ edit_upload_metadata(
'upload_id': 'dcvqjWYgSZeDVLaJkL6E7w',
'upload_name': 'NOMAD Tutorial, Prepare DFT example for sharing using API',
'upload_create_time': '2026-09-29T09:47:05.108000Z',
'main_author': 'f250f5ab-b05c-4bad-9939-5f4883c7a694',
'main_author': '00000000-0000-4000-8000-000000000001',
'coauthors': [],
'coauthor_groups': [],
'reviewers': [],
'reviewer_groups': [],
'writers': ['f250f5ab-b05c-4bad-9939-5f4883c7a694'],
'writers': ['00000000-0000-4000-8000-000000000001'],
'writer_groups': [],
'viewers': ['f250f5ab-b05c-4bad-9939-5f4883c7a694'],
'viewers': ['00000000-0000-4000-8000-000000000001'],
'viewer_groups': [],
'published': False,
'published_to': [],
Expand Down Expand Up @@ -577,8 +591,8 @@ for user in candidates:
??? success "Example notebook output"

```
Found the user 'Jane Doe' (username 'jane.doe') with user_id='13b845c3-e48d-4234-8c51-4f88c6897ac8'
Found the user 'Jane Doe' (username 'jane.doe@example.org') with user_id='ebb26223-0cec-4d81-98f5-3b25db945b54'
Found the user 'Jane Doe' (username 'jane.doe') with user_id='00000000-0000-4000-8000-000000000002'
Found the user 'Jane Doe' (username 'jane.doe@example.org') with user_id='00000000-0000-4000-8000-000000000003'
```

If several users have the same name, use the username to pick the right one.
Expand Down Expand Up @@ -655,15 +669,15 @@ pprint(response)

```
{'data': {'coauthor_groups': [],
'coauthors': ['ebb26223-0cec-4d81-98f5-3b25db945b54'],
'coauthors': ['00000000-0000-4000-8000-000000000003'],
'complete_time': '2026-09-29T11:18:29.488000Z',
'current_process': '_publish_upload',
'embargo_length': 3,
'entries': 1,
'errors': [],
'last_status_message': 'Process completed successfully',
'license': 'CC BY 4.0',
'main_author': 'f250f5ab-b05c-4bad-9939-5f4883c7a694',
'main_author': '00000000-0000-4000-8000-000000000001',
'process_running': True,
'process_status': 'PENDING',
'published': False,
Expand All @@ -676,16 +690,35 @@ pprint(response)
'upload_name': 'NOMAD Tutorial, Prepare DFT example for sharing '
'using API',
'viewer_groups': [],
'viewers': ['f250f5ab-b05c-4bad-9939-5f4883c7a694',
'ebb26223-0cec-4d81-98f5-3b25db945b54'],
'viewers': ['00000000-0000-4000-8000-000000000001',
'00000000-0000-4000-8000-000000000003'],
'warnings': [],
'with_embargo': True,
'writer_groups': [],
'writers': ['f250f5ab-b05c-4bad-9939-5f4883c7a694',
'ebb26223-0cec-4d81-98f5-3b25db945b54']},
'writers': ['00000000-0000-4000-8000-000000000001',
'00000000-0000-4000-8000-000000000003']},
'upload_id': 'dcvqjWYgSZeDVLaJkL6E7w'}
```

This code triggers the publication action for the DFT project and prints the server response confirming the operation.
The response confirms that the publication request was accepted, not that publication succeeded. Publication runs asynchronously, so wait for it to finish and inspect its final status in a separate cell:

```python
published_upload = wait_for_process(upload_id=dft_upload_id, url='test')
print('Process status:', published_upload.process_status)
print('Published:', published_upload.published)
print('Errors:', published_upload.errors)
print('Warnings:', published_upload.warnings)
```

??? success "Final publication status"

```
Process status: SUCCESS
Published: True
Errors: []
Warnings: []
```

Confirm that the process status is `SUCCESS`, `Published` is `True`, and `Errors` is empty; review any warnings as well. For details, see [Inspect a project](#inspect-a-project).

<!-- TODO: Consider a step that assigns a DOI to the published project via the API (POST /uploads/{upload_id}/action/assign-doi), mirroring "Assign a DOI to your project" in upload_publish.md. The test deployment used in this tutorial cannot assign DOIs (DataCite is disabled there), and nomad-utility-workflows 0.3.2 has no helper for it. -->