Repository navigation
Fix slurm restd attr - #1226
Merged
Merged
Fix slurm restd attr#1226
Conversation
…update-metadata-correctly
sbzeytun
force-pushed
the
fix-slurm-restd-attr
branch
from
June 16, 2026 14:37
45b680a to
b1d2eff
Compare
…lurm-restd-attr Accidentally made commit before pulling changes that github made to coverage.svg
Contributor
|
|
Contributor
|
|
pagrubel
requested changes
Aug 19, 2026
Collaborator
There was a problem hiding this comment.
It's nice to see the current running time when using slurmrestd.
However, the runtime resets to zero if the jobs is cancelled or fails or possibly times out.
I was able to test it for cancelling and failed by running the clamr example and cancelling the job with the scancel command. Also, tested the failed state by deleting the graphics_output directory while ffmpeg was running. I haven't tested timeout but could come up with a test for that. Here are the results:
The query showed ffmpeg at 9 seconds before it failed, show should show that.
Archived/Partial-Fail
task_name task_state job_id partition runtime nodes command
----------- ------------ -------- ----------- --------- ------- --------------------------------------
----------------------------------------------
clamr COMPLETED 17188568 scaling 00:01:54 cn336 /vast/home/pagrubel/beeworkdir/clamr-f
fmpeg-build/workdir/clamr-86ef/clamr-86ef.sh
ffmpeg FAILED 17188569 scaling 00:00:00 cn366 /vast/home/pagrubel/beeworkdir/clamr-f
fmpeg-build/workdir/ffmpeg-bd91/ffmpeg-bd91.sh
Similarly, for this example the time was over a minute for clamr before I cancelled it.
Think about a job that runs for hours and somehow is cancelled or fails. The user may need that information.
beeflow query 76
Archived/Failed
task_name task_state job_id partition runtime nodes command
----------- ------------ -------- ----------- --------- ------- ----------------------------------------------------------------------------------
clamr CANCELLED 17187581 scaling 00:00:00 cn339 /vast/home/pagrubel/beeworkdir/clamr-ffmpeg-build/workdir/clamr-74cd/clamr-74cd.sh
ffmpeg DEP_FAIL
Contributor
|
|
pagrubel
previously requested changes
Aug 20, 2026
pagrubel
left a comment
pagrubel
left a comment
Collaborator
There was a problem hiding this comment.
I tested this with a timelimit and a longer clamr example and it also clears out all time for the TIMEOUT state. There may be other states that this will not work for such as PAUSED or SUSPENDED.
pagrubel
dismissed
their stale review
August 20, 2026 23:33
Fixed with the latest commit, no longer depends on the state. This will work for other situations such as TIMEOUT and paused jobs.
Contributor
|
|
pagrubel
approved these changes
Aug 21, 2026
pagrubel
left a comment
pagrubel
left a comment
Collaborator
There was a problem hiding this comment.
This looks good, I did make a change to calculate time for any state.
Merged
pagrubel
added a commit
that referenced
this pull request
Aug 26, 2026
* Start pre-release 0.1.13 * Start pre-release 0.1.13dev0 * fix typo * Don't update coverage.svg until merged into develop (#1240) * Don't update coverage.svg until merged into develop * Add badge to pr comments (may not be available for external PR's) * Add caution to check coverage before pushing to main in a release * Make it so that it fails if coverage is under 65 --------- Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Patrica Grubel <pagrubel@lanl.gov> * Bump setuptools from 75.3.0 to 78.1.1 (#1239) Bumps [setuptools](https://github.com/pypa/setuptools) from 75.3.0 to 78.1.1. - [Release notes](https://github.com/pypa/setuptools/releases) - [Changelog](https://github.com/pypa/setuptools/blob/main/NEWS.rst) - [Commits](pypa/setuptools@v75.3.0...v78.1.1) --- updated-dependencies: - dependency-name: setuptools dependency-version: 78.1.1 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Move coverage badge to gh-pages (#1245) * Move coverage badge to gh-pages * Make a new comment each time coverage changes --------- Co-authored-by: Sare Beste Zeytun <sbzeytun@arizona.edu> * Build docs after badge gets updated in gh-pages (#1247) * Build docs after badge gets updated in gh-pages --------- Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> * Fix main docs publication (#1250) * Fix main docs publication * update version to 0.1.13dev1 * Update version on README.rst to 0.1.13dev1 * Update README.rst one more fix for dev1 * Optimize CI tests through github's caching mechanism (#1230) * Optimize CI tests through github's caching mechanism * Organize dependencies to make caching work * Separate into different caches * Cache containers if they are unchanged * Test to see if cache speeds up --------- Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> * optional cloud dependencies (#1251) * remove optional cloud dependencies * Get rid of poetry information that causes issues (#1255) Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> * docs: add top-level AGENTS.md for AI coding agents (#1256) Add an AGENTS.md at the repository root to give AI coding agents (and humans) a concise, project-specific guide for working in the BEE codebase. The document covers: - Project overview (BEE as a CWL-based HPC/cloud workflow orchestrator using Neo4j and Slurm/Flux) - Repository layout for the beeflow package and supporting directories - Environment and tooling (Python >=3.11,<=3.14, Poetry, the beeflow CLI) - Testing with pytest, mirroring ci/unit_tests.sh, and notes on integration tests - Code style and linting via pylint (99-char lines) and the pre-commit hooks - Documentation build steps with Sphinx - Guidelines for agents (conventions, dependency handling, testing and linting expectations, and the develop default branch) Content is derived from pyproject.toml, setup.cfg, .pre-commit-config.yaml, README.rst, ci/unit_tests.sh, and docs/README.md so the guidance matches this project. Signed-off-by: Howard Pritchard <howardp@lanl.gov> * Update AGENTS.md file with correct poetry info (#1258) * Ceanup caches on closed prs (#1253) * Make cache keys more descriptive and cleanup caches on closed prs --------- Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> * Fix slurm restd attr (#1226) * Update metadata correctly when querying and save metadata as yaml #1192 * Fix yaml formatting issue #1223 * Add runtime as default attribute for slurmrestd * calculate slurmrestd runtime from available timestamps --------- Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Patricia Grubel <pagrubel@lanl.gov> * working version of querying latest workflow (#1238) * Allow user to query the latest workflow with the command "beeflow query" (leaving off the id) * fixed pytest errors by allowing match_shot_id to accept an optional wf_id * add tests that cover cases for query() and match_short_id() * updated docs from wf_id[required] to wf_id[optional] * Add the workflow name and short id to the top of the query data --------- Co-authored-by: Santiago Lopez <santiagolopez@lanl.gov> * Allow baremetal jobs (#1218) * first step: modify task manager worker selection * add cat-grep-tar-baremetal example * add to documentation * add unit tests for new scheduler per task behavior * update simple worker status directory path * make sure tm db is keeping track of scheduler and type error fix * add WorkloadRequirement to cwl python API and add testing * modify example 1 to show baremetal cwl requirement * add cwl syntax check for workloadRequirement during parsing * Add a simple baremetal example for cwl generator --------- Co-authored-by: Patricia Grubel <pagrubel@lanl.gov> (cwl example) * RELEASE 0.1.13 (#1261) --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Howard Pritchard <howardp@lanl.gov> Co-authored-by: Sare Beste Zeytun <sbzeytun@arizona.edu> Co-authored-by: Sare Beste Zeytun - 407210 <szeytun@darwin-fe2.lanl.gov> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Howard Pritchard <hppritcha@gmail.com> Co-authored-by: Santiago Lopez <santiago2@uchicago.edu> Co-authored-by: Santiago Lopez <santiagolopez@lanl.gov> Co-authored-by: Krishna Chilleri <krishnachilleri@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes issue #1225. I also made the time specific attributes to be consistent among the slurmrestd and slurm command line schedulers, which was something I was wanting to do last time.