Pipelines — repeatable preparation | ATX AnalyticsTable Handbook

Pipelines — repeatable preparationThe difference between cleaning it every morning and the cleaning being part of the refresh. The pipeline designer is where one-off cleanup becomes a repeatable recipe: an ordered list of steps — sort, dedupe, merge columns, value mapping, text cleanup, anomaly replacement, keep or drop columns, clear formatting, mask personal data, run SQL — saved with the project and bound to one dataset. The commercial point is a single sentence: refreshing a pinned region automatically re-applies every pipeline bound to it. A one-off ribbon transform is lost on the next refresh; the same operation as a pipeline step survives.

What it does

Ordered steps covering reshaping, cleanup, column selection, formatting, privacy and raw SQL
Run, Refresh & run, Step through with preview, Undo last run, and Apply to another dataset with column remapping
Bound to a static pin or a named grid range, with steps validated against the bound dataset's columns
Automatic re-application on every refresh of the bound region
Each pipeline stored as its own Git-friendly JSON file
Every run appends an immutable line to an append-only run log
Ask Greg can propose a whole pipeline or add one step to an existing one — dry-run with and without the step before you place it, and never changing the steps already there

How it works

Build the recipe once against real data, stepping through with preview until each step does what you meant.
Bind it, then use Refresh & run as the daily button — the source reloads and the recipe re-applies in one action.
Errors are deliberately asymmetric: dropping a column that is absent is a note, because the same recipe often runs against slightly different shapes — but keeping only columns that do not exist fails loudly and lists what is actually available, because the alternative would wipe the table.
Write-back depends on the binding: a pinned region genuinely narrows, while a grid range has removed columns blanked in place — a pipeline never inserts or deletes sheet columns.
The Clear formatting step exists because reshaping silently invalidates the formatting sitting on the data; a colour scale calibrated to last month still looks authoritative while describing numbers that no longer exist.

Practices that keep it reliable

Promote every repeated manual cleanup into a pipeline step — that is where the automation dividend is
Step through a new pipeline against a copy before binding it to the production region
Review the run log after the first scheduled week; it answers 'did it actually run?' without guesswork

In a regulated environment

Pipelines are reviewable artefacts: a .pipeline.json file in the repository, changed by pull request like anything else.
The append-only run log records what ran, when and against what — the evidence an internal audit asks for.
No secret material is written into a pipeline file, so it can be shared and reviewed without exposure.

Screens

The pipeline designer — an ordered recipe, bound to one dataset and re-run on every refresh.
A pipeline-prepared sheet: the cleanup is part of the refresh, not a morning chore.

Related guides

ATX Institute — home

Products

Company

Resources

Guides

ATX Institute on LinkedIn

This site needs JavaScript for the full experience. Every page above is reachable without it.