Trim sisiphys -> gaia kafka msgs down to:
step-startstep-complete
- Compute next tasks to run and workflow complete/stalled/continues once per task completion, not per file data-complete.
- This will save time, esp. at scale, and make STEP COMPLETE appear in the log before WORKFLOW COMPLETE.
step-error -- puts the task in state :errortask-error -- puts the task in state :exception
- What's the difference? Merge this with
step-error or rename it to step-exception.
- Does Gaia need to distinguish
step-terminated and step-timeout?
Check that Gaia and Sisyphus match up in all the Kafka messages, including when some part of a task fails.
Trim sisiphys -> gaia kafka msgs down to:
step-startstep-completestep-error-- puts the task in state:errortask-error-- puts the task in state:exceptionstep-erroror rename it tostep-exception.step-terminatedandstep-timeout?Check that Gaia and Sisyphus match up in all the Kafka messages, including when some part of a task fails.