Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions rust/crates/sift_cli/assets/docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- [ULog](./data/import-ulog.md)
- [Backups](./data/import-backups.md)
- [Exporting Data](./data/exporting.md)
- [Jobs](./data/jobs.md)

# Reference

Expand Down
59 changes: 59 additions & 0 deletions rust/crates/sift_cli/assets/docs/src/data/jobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Jobs

Server-side work in Sift — data imports, exports, and rule evaluations — runs
as a job. The `get`, `status`, and `wait` commands inspect and poll those jobs,
which is what makes the CLI usable in CI/CD.

## Why not just pass `--wait`?

`sift-cli import ... --wait` blocks until the job finishes. That is the right
default for interactive use, but it serializes work: firing five imports means
five sequential blocking commands. And it does not compose — you cannot start
several imports in parallel, do other work, then confirm they all landed at
the end.

The verb-first commands split those two concerns:

1. **Kick off work without waiting.** `sift-cli import ...` (no `--wait`)
uploads the file and prints the assigned job ID. It exits `0` even if the
server-side job later fails.
2. **Poll the jobs later.** `sift-cli wait job <ID> [ID ...]` blocks until
every named job reaches a terminal state and exits non-zero if any failed.

## The CI/CD pattern

```sh
# Fire imports in parallel; capture each job id.
JOB_A=$(sift-cli import csv a.csv --asset engine | awk '/Job ID/ {print $NF}')
JOB_B=$(sift-cli import csv b.csv --asset engine | awk '/Job ID/ {print $NF}')
JOB_C=$(sift-cli import csv c.csv --asset engine | awk '/Job ID/ {print $NF}')

# Gate the pipeline on all three finishing successfully.
sift-cli wait job "$JOB_A" "$JOB_B" "$JOB_C"
```

The `import` command prints a `Job ID: <uuid>` line on the no-wait path
specifically so scripts can capture it.

## Inspecting jobs

- `sift-cli get jobs` lists the 50 most recent jobs, newest first. Add
`--job-type` (`data-import`, `data-export`, `rule-evaluation`) or `--status`
(`created`, `running`, `finished`, `failed`, `cancelled`, `cancel-requested`)
to narrow it. `--limit` overrides the page size.
- `sift-cli get job <ID>` prints the full details on one job: type, status,
timestamps, and failure details when the job failed.
- `sift-cli status job <ID>` is the scripting form. It prints one status word
to stdout and exits with a code that reflects the job state.

### Exit codes for `status job`

| Code | Meaning |
| ---- | ------------------------------------------ |
| `0` | Job finished successfully. |
| `1` | Job failed. |
| `2` | Job was cancelled or cancel is requested. |
| `3` | Job is still running or has not started. |

`wait job` uses `0` if every job finished and `1` if any job failed or was
cancelled; per-job status is printed to stderr for the non-success cases.
12 changes: 12 additions & 0 deletions rust/crates/sift_cli/assets/docs/src/reference/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,18 @@ Export data from Sift. See [Exporting Data](../data/exporting.md).
Verify credentials and connectivity. See
[Verifying Your Setup](../getting-started/verifying.md).

## `get`, `status`, `wait`

Inspect and poll Sift resources. See [Jobs](../data/jobs.md) for the CI/CD
pattern (fire imports without `--wait`, then gate on a single `wait job`).

| Command | Description |
| ------------------------------------------ | ---------------------------------------------------------------------------------- |
| `get jobs [--job-type ...] [--status ...]` | List recent jobs, newest first. |
| `get job <JOB_ID>` | Show full details for one job. |
| `status job <JOB_ID>` | Print a terse status line. Exit `0` finished, `1` failed, `2` cancelled, `3` running. |
| `wait job <JOB_ID> [JOB_ID ...]` | Block until every named job reaches a terminal state. |

## `install`

Install optional tooling.
Expand Down
7 changes: 7 additions & 0 deletions rust/crates/sift_cli/assets/skills/sift/references/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Key subcommands:
- `import`: `csv`, `parquet flat-dataset`, `parquet cpr`, `tdms`, `hdf5`,
`ulog`, `backups`.
- `export`: `run`, `asset` (to CSV and other formats).
- `get`, `status`, `wait`: inspect and poll server-side jobs (`get jobs`,
`get job <ID>`, `status job <ID>`, `wait job <ID> [ID ...]`).
- `mcp`: start the MCP server.
- `ping`: verify credentials and connectivity.
- `config`: manage profiles and credentials.
Expand Down Expand Up @@ -72,6 +74,11 @@ per session. The rest apply to each subcommand invocation.
it you cannot confirm the data actually landed. Relay the final stdout
line to the user verbatim. `import backups` is the one exception: it
accepts no `--wait`, `--preview`, or `--run`.

Skip `--wait` only when firing multiple imports in parallel. In that
case capture each `Job ID: <uuid>` line from the upload output, then
run `sift-cli wait job <ID> [ID ...]` to block on the whole batch and
report failures. `wait job` exits non-zero if any job failed.
7. **Surface the Explore link from import output.** Each profile must set
`app_uri`. `sift-cli import` prints a `View in Sift: <URL>` tip when this
value is usable. Surface the URL as plain text, in full. Do not wrap it in a
Expand Down
91 changes: 91 additions & 0 deletions rust/crates/sift_cli/src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,97 @@ pub enum Cmd {
#[cfg(feature = "mcp")]
#[command(hide = true)]
Mcp(McpArgs),

/// Retrieve Sift resources (e.g. jobs)
#[command(subcommand)]
Get(GetCmd),

/// Print a terse status line for a resource; exit code reflects state
#[command(subcommand)]
Status(StatusCmd),

/// Block until one or more resources reach a terminal state
#[command(subcommand)]
Wait(WaitCmd),
}

#[derive(Subcommand)]
pub enum GetCmd {
/// List recent jobs, newest first
Jobs(GetJobsArgs),

/// Show full details for a single job
Job(GetJobArgs),
}

#[derive(clap::Args)]
pub struct GetJobsArgs {
/// Filter by job type
#[arg(long, value_enum)]
pub job_type: Option<JobTypeArg>,

/// Filter by job status
#[arg(long, value_enum)]
pub status: Option<JobStatusArg>,

/// Max jobs to return (clamped to the server's cap)
#[arg(long, default_value_t = 50)]
pub limit: u32,
}

#[derive(clap::Args)]
pub struct GetJobArgs {
/// Job ID
pub job_id: String,
}

#[derive(Subcommand)]
pub enum StatusCmd {
/// Print the current status of a job. Exit code: 0 finished, 1 failed,
/// 2 cancelled or cancel-requested, 3 still running
Job(StatusJobArgs),
}

#[derive(clap::Args)]
pub struct StatusJobArgs {
/// Job ID
pub job_id: String,
}

#[derive(Subcommand)]
pub enum WaitCmd {
/// Block until every named job reaches a terminal state. Exit 0 if all
/// finished; nonzero if any failed or was cancelled
Job(WaitJobArgs),
}

#[derive(clap::Args)]
#[command(
override_usage = "sift-cli wait job <JOB_ID> [JOB_ID ...]",
after_help = "Example:\n \
sift-cli wait job <JOB_ID_1> <JOB_ID_2> <JOB_ID_3>"
)]
pub struct WaitJobArgs {
/// One or more job IDs to wait on, separated by spaces
#[arg(required = true, value_name = "JOB_ID")]
pub job_ids: Vec<String>,
}

#[derive(clap::ValueEnum, Clone, Debug)]
pub enum JobTypeArg {
DataImport,
DataExport,
RuleEvaluation,
}

#[derive(clap::ValueEnum, Clone, Debug)]
pub enum JobStatusArg {
Created,
Running,
Finished,
Failed,
Cancelled,
CancelRequested,
}

#[cfg(feature = "mcp")]
Expand Down
101 changes: 47 additions & 54 deletions rust/crates/sift_cli/src/cmd/import/mod.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use anyhow::Result;
use crossterm::style::Stylize;
use std::{process::ExitCode, time::Duration};
use tokio::time::sleep;
use std::process::ExitCode;

use sift_rs::{SiftChannel, common::r#type::v1::ChannelConfig, jobs::v1::JobStatus};

Expand Down Expand Up @@ -40,6 +39,7 @@ pub async fn finish_import(
if !wait {
Output::new()
.line(format!("{} file for processing", "Uploaded".green()))
.line(format!("{}: {job_id}", "Job ID".green()))
.tip(pending_import_tip(
&target.location,
target.explore_url.as_deref(),
Expand All @@ -60,72 +60,65 @@ pub async fn wait_for_job_completion(
let spinner = Spinner::new();
spinner.set_message(format!("{} file for processing", "Uploaded".green()));

let mut job_service = JobServiceWrapper::new(grpc_channel.clone());
let mut job_service = JobServiceWrapper::new(grpc_channel);

let Some(mut job) = job_service.get_job(&job_id).await? else {
spinner.finish_and_clear();
let outcome = job_service
.poll_until_terminal(&job_id, |job| match job.job_status() {
JobStatus::Running => {
spinner.set_message(format!("{} imported file", "Processing".green()));
}
JobStatus::CancelRequested => {
spinner.set_message(format!(
"{} was requested but the job may still finish",
"Cancellation".green()
));
}
_ => (),
})
.await?;

spinner.finish_and_clear();

let Some(job) = outcome else {
Output::new()
.line("The file was successfully uploaded but the job was unexpectedly not found")
.tip("Please notify Sift about this bug")
.eprint();
return Ok(ExitCode::FAILURE);
};

loop {
sleep(Duration::from_secs(3)).await;

let Some(updated_job) = job_service.get_job(&job.job_id).await? else {
spinner.finish_and_clear();
match job.job_status() {
JobStatus::Finished => {
let mut tip_text =
format!("The data should be available on the {import_output_location}");
tip_text.push_str(&explore_or_note(explore_url.as_deref()));
Output::new()
.line(format!("{} data import job", "Completed".green()))
.tip(tip_text)
.print();
Ok(ExitCode::SUCCESS)
}
JobStatus::Cancelled => {
Output::new()
.line(format!("{} data import job", "Cancelled".green()))
.print();
Ok(ExitCode::SUCCESS)
}
JobStatus::Failed => {
Output::new()
.line("The file was successfully uploaded but the job was unexpectedly not found")
.line("Processing failed")
.tip("Please check the Sift jobs manage page for further details")
.eprint();
Ok(ExitCode::FAILURE)
}
other => {
Output::new()
.line(format!("unexpected job status `{other:?}`"))
.tip("Please notify Sift about this bug")
.eprint();
return Ok(ExitCode::FAILURE);
};
job = updated_job;

match job.job_status() {
JobStatus::Created => (),
JobStatus::Running => {
spinner.set_message(format!("{} imported file", "Processing".green()));
}
JobStatus::CancelRequested => {
spinner.set_message(format!(
"{} was requested but the job may still finish",
"Cancellation".green()
));
}
JobStatus::Cancelled => {
spinner.finish_and_clear();
Output::new()
.line(format!("{} data import job", "Cancelled".green()))
.print();
break;
}
JobStatus::Failed => {
spinner.finish_and_clear();
Output::new()
.line("Processing failed")
.tip("Please check the Sift jobs manage page for further details")
.eprint();
return Ok(ExitCode::FAILURE);
}
JobStatus::Finished => {
spinner.finish_and_clear();
let mut tip_text =
format!("The data should be available on the {import_output_location}");
tip_text.push_str(&explore_or_note(explore_url.as_deref()));
Output::new()
.line(format!("{} data import job", "Completed".green()))
.tip(tip_text)
.print();
break;
}
_ => (),
Ok(ExitCode::FAILURE)
}
}
Ok(ExitCode::SUCCESS)
}

pub struct TimePreview<'a> {
Expand Down
Loading
Loading