Runs
Mark a test, flight or shift as a run, set pass criteria, score it, and compare it with other runs.
Runs and pass criteria
A run is a named stretch of time on one device: a test, a flight, a shift. Open it when the work starts, close it when it ends, and say up front what a good one looks like.
Why
Telemetry is one long stream. The things you care about are pieces of it: the third hotfire, this morning's flight. Without a name and a start and end, those pieces are gone once you scroll away.
Writing down the limits before the test matters too. Deciding what "passing" means after you have seen the data is how bad runs get waved through.
What a run holds
- A name and the device it belongs to.
- A start, and an end once it closes.
- A status: active while it runs, then completed or aborted.
- Optional tags and metadata, like build or operator.
- Optional pass criteria: a metric, an operator (>, >=, <, <=, =, !=) and a value, with an optional label.
Opening a run
In the app, press R on the Runs page and choose Start now. The run opens at that moment and stays open until you click End run on its page. You can add pass criteria in the same dialog.
From a bench script, wrap the test in px.run(). A clean exit closes the run as completed. An exception closes it as aborted and re-raises. If the device has never sent data before, Plexus creates it when the run opens.
from plexus import Plexus
px = Plexus(api_key="plx_...", source_id="pod-7")
with px.run("hotfire-03", pass_criteria=[
{"metric": "motor.temp_c", "operator": "<", "value": 85,
"label": "motor stays cool"},
{"metric": "frames.dropped", "operator": "=", "value": 0},
]) as run:
bench.execute()Limits
Runs are part of the paid plans. Pass criteria can be edited while a run is active and are locked once it closes, so a finished run always matches the test it was judged against.
Compare runs
Lay one run over another, lined up at the moment each started. Differences show up as two traces pulling apart on the same chart.
How it works
Plexus shifts the other run's data so its start lines up with the start of the run you are looking at. That other run is drawn as a ghost, in its own color, on every chart that reads the same metrics. A small legend on each chart names both runs.
- Ghosts are drawn on time charts. Bar charts and charts that plot one metric against another are skipped, since shifting time means nothing there.
- Hover still reads every trace, so you can compare values at the same moment into each run.
Using it
From a run's page, click Compare and pick another run. That opens a compare view with two run pickers, a swap button, and charts for the device's metrics (the first 12). No dashboard needed.
On a dashboard, open the Runs menu in the toolbar. Enter opens a run, which frames the dashboard on its window. Space overlays it instead. You can overlay several runs at once, each in its own color: orange, yellow, pink, teal, lime. Esc exits compare and puts the dashboard back exactly where it was.
Runs list and run pages
Runs got their own page. Every test, flight or shift is listed with its device, status, duration and result, and the one that is running now sits at the top.
The runs list
The Runs page lists every run in the org with its name, device, status, duration, start time and result. Search by name, or filter by status: active, completed or aborted.
Any run that is still going gets a green banner at the top of the list, so the live test is one click away. Hover a row to compare it with another run, or to replay its window in one of your dashboards. Right-click a row for edit and delete.
The run page
Open a run and you get two tabs. Overview holds the properties (device, status, start, end, duration, who made it), the result, the pass criteria and any annotations made during the run. Charts draws the device's metrics across the run's window.
While a run is active, the page shows elapsed time and an End run button. Rename it in place and save with Cmd+S. Delete it from the danger zone at the bottom.
Runs on the timeline
Runs also show up as labeled bands on the dashboard's time scrubber. Click one to frame the dashboard on that run. A run you make anywhere appears there right away.
Pass/fail results and export for runs
A run that declared pass criteria gets a verdict the moment it closes: pass or fail, with the number that decided each limit. Any run with a device can be exported as CSV.
How a run is scored
When a run goes from active to closed, Plexus checks each criterion against every sample of that metric in the run's window. The rules are strict on purpose.
- A criterion holds only if every sample satisfies it. One excursion fails it.
- A metric with no samples in the window fails. "We never measured it" is not "it was fine".
- The run passes only if every criterion passes. A run with no criteria gets no verdict, not a free pass.
- Just-closed runs wait up to 4 seconds so the last points can land before scoring. Scoring gives up after 8 seconds and records an error, so a slow query never stops a run from closing.
Reading the verdict
The run page's Overview shows Passed or Failed with a count, like 3/4. Failures sort to the top, and each line says what decided it: "reached 91.4, limit is < 85" or "peak 72.1 stayed under 85". The runs list shows PASS or FAIL in its Result column.
From code, px.end_run() hands the verdict back in test_result, so a bench script can act on it.
run = px.start_run("hotfire-03", pass_criteria=[
{"metric": "motor.temp_c", "operator": "<", "value": 85,
"label": "motor stays cool"},
])
bench.execute()
run = px.end_run(run)
print(run["test_result"]["passed"])Export
Export on the run page downloads a CSV of every metric the device reports over the run's window, not just the ones charted. Rows are timestamp, metric, value. Channels sampled on their own clocks stay honest that way, and pivoting is one line in pandas.
Each metric is capped at 100,000 points. If a run holds more, Plexus tells you and suggests exporting a shorter window.
Create a run from a chart selection
Shift-drag across any chart to select a stretch of time. Plexus offers to keep that window as a run, already tied to the device the chart was reading.
How it works
Shift-drag on a chart still zooms to the part you selected, as it always has. Now a Create run button also appears on that chart once you let go.
Click it and the create dialog opens straight on the window step. Start and end are filled in from your selection, down to the second, and the dialog tells you they came from the window you dragged. The device is taken from the chart, so the new run knows where its data lives.
- Adjust either end if you need to. The dialog checks the window as you type: no end before the start, no zero-length windows.
- Add pass criteria if you want: a metric, an operator (>, >=, <, <=, =, !=) and a value. The metric field suggests names the device actually reports.
- The run is created already closed, because both ends are known. You stay on the dashboard and keep working.
The other way in
The Runs menu in the dashboard toolbar has a Create run row that reads "from this window". It uses whatever the dashboard is showing, including quick ranges like the last 15 minutes. If the dashboard reads from exactly one device, that device is picked for you. If it reads from several, you choose.
On the Runs page itself, press R to open the same dialog.
Limits
The Create run button does not appear on shared or public dashboard links, because anonymous viewers cannot create runs. Runs are part of the paid plans. On the Free plan the dialog shows the upgrade prompt instead of an error.