How to Turn RTF and TXT Tool Files Into Structured Data

Why RTF and TXT tool files are deceptively hard

Open one in a text editor and it looks tame. A header, some labels, a column of measurements. The trouble is that "looks readable to a human" and "parses reliably by machine" are two very different bars, and tool exports clear the first one while failing the second in ways that only show up at scale.

RTF wraps control codes around your real numbers

RTF is a formatting language, not a data format. A single measured value can arrive surrounded by font tables, color tables, and escape sequences. A minus sign might be a literal hyphen on one line and an escaped character on the next. A degree symbol shows up as an escaped code point. If you write a naive regex that grabs digits, you will happily capture a font size or a color index and treat it as a reading. The number you want is real; the noise around it is the problem.

The layout changes per routine and per firmware

The same machine, running a different measurement routine, emits a different layout. Then the vendor pushes a firmware update and the header gains a line, a column shifts, or a label gets renamed from "Dev" to "Deviation." Any parser that counts characters or assumes fixed column positions will silently drift. This is the single biggest reason a parser that worked in a demo fails in production — the input was never as stable as it looked on the day you tested it.

Mixed encodings and stray bytes

One file is Windows-1252, the next is UTF-8, an older one has a byte-order mark glued to the front. A micro sign or a plus-minus symbol renders fine to your eye and then blows up a strict decoder. Occasionally a machine writes a partial buffer and you get a stray null byte in the middle of an otherwise good line. None of this is exotic — it is just the reality of files written by embedded software over many years.

Corrupt and partial lines are normal, not rare

A probe times out mid-write. The network share hiccups and truncates the file. A tech aborts a run and the last record is half-formed. In a batch of a few thousand files, you will always find some that are broken. A parser that assumes every line is well-formed does not just miss those lines — it usually crashes on the first bad one and takes the whole batch down with it.

A pragmatic step-by-step approach

The goal is not a clever one-liner. It is a small pipeline where each stage does one job and hands clean output to the next. This is the same shape we use for broader manufacturing data mapping work, scaled down to a single file format.

1. Decide what a "record" is

Before extracting anything, define the unit of data. Is one record a single measured feature, or is it a whole part with many features? Find the anchor that starts each record — a line that begins with a part ID, a timestamp, or a known keyword. Split the file on that anchor rather than on line count. Get this wrong and everything downstream inherits the mistake.

2. Extract the fields

For RTF, strip the formatting layer first — decode the escapes and drop the control words — so you are working against the underlying text, not the markup. Then pull fields by their labels, not by their positions. Match "Nominal," "Actual," and "Tol" by name and capture the value that follows. Label-based extraction survives the column shifts that position-based extraction cannot.

3. Normalize units and types

A raw "0.0021" is a string until you make it a number, and a number is meaningless until you know its unit. Convert everything to a consistent unit and a consistent type at this stage — inches to millimeters, timestamps to a single time zone, blanks to explicit nulls. Do the conversion once, here, so no downstream report has to guess.

4. Validate against expected ranges

Every field has a plausible range. A bore diameter is not negative and not a mile wide. A temperature reading of 9999 is a sensor fault code, not a measurement. Check each value against what is physically reasonable, and flag anything that fails. This is where you catch the garbage that survived extraction looking superficially valid.

5. Quarantine failures with a reason

When a record cannot be parsed or fails validation, do not discard it and do not crash. Write it to a quarantine bucket with a note: "unreadable encoding on line 40," "actual value out of range," "record truncated." A human can then review the pile, and you can see at a glance whether a new failure pattern just appeared — which is usually your early warning that a machine's firmware changed.

Why the one-off script breaks in week two

A script written against three sample files will parse those three files beautifully. It assumes the encoding it saw, the layout it saw, and that every line is complete. Then week two arrives with a file from a different routine, a firmware update, or a truncated run, and the script either throws an exception or — worse — quietly produces wrong numbers that look right. The failure is not that the author was careless. It is that clean input was assumed where messy input was guaranteed.

A durable parser is built the opposite way. It expects mess. It decodes defensively, matches by label, validates every value, quarantines what fails instead of dying on it, and logs enough that when something new breaks you can see exactly which file and which line caused it. It treats new failure modes as data to learn from, not as crashes to firefight. That resilience is the whole difference between a script and a system, and it is what data mapping for CNC machine data or injection molding data is really about.

Hand-copying versus automating

Not every pile of files justifies a parser. If someone opens a handful of reports a week and types a few numbers into a sheet, that is cheaper than building anything, and you should leave it alone. Automation earns its keep when the volume is high, the copying is error-prone, the data feeds decisions that matter, or the same person is doing the same tedious transcription every single day.

The tell is usually a mix of pain and frequency. A hundred files a day being retyped by hand is both slow and a source of quiet errors, and those errors surface later as bad yield numbers or a scrapped lot nobody can explain. When copying is on the critical path of a decision, the case for building is strong. When it is a rare, low-stakes chore, it is not.

A short, realistic example

Picture a batch of RTF files from a coordinate measuring machine. Most records are clean: a feature name, a nominal value, an actual value, a deviation, and a pass or fail flag. Your pipeline splits on the feature-name anchor, strips the RTF control words, matches "Nominal" and "Actual" by label, converts the values to millimeters, checks each against its tolerance band, and writes a tidy row. Thousands of good measurements flow through without a hitch.

Then one record reads "Actual: TIMEOUT" where a number should be — the probe never made contact and the machine wrote a fault string instead of a reading. A naive script either stores "TIMEOUT" as a value or crashes trying to convert it to a number. The durable parser catches it at the validation step, quarantines that one record with the reason "non-numeric actual — probe timeout," and keeps processing the rest of the file. At the end you have clean structured data for every good measurement, plus a short, honest list of exactly what could not be read and why. That list is not a failure. It is the parser telling you the truth about your data.