A pipeline that strips hidden junk from every new file, run every morning. A 1:25 full-HD video recorded on the real ATX AnalyticsTable application, part of the pipelines and data quality tutorials.
A supplier file that looks clean can still fail every lookup: a zero-width space inside a product code, a non-breaking space between words, & where an ampersand should be, "emea " with a trailing blank. TRIM and CLEAN in Excel do not catch all of them, and fixing them by hand has to be repeated with every new file.
This tutorial builds the fix once. The imported block is named as a range, supplier_file, and a pipeline with one Replace anomalies step is bound to it — hidden spaces and zero-width characters are ticked by default; HTML entities and leading / trailing whitespace are added. Step through runs it as a preview, nothing written, before Apply to data.
Finally the pipeline is scheduled for every weekday at 06:00, so each morning's file is cleaned before anyone opens it.
This site needs JavaScript for the full experience. Every page above is reachable without it.