How to measure spend classification accuracy

Published 3 September 2026 · 6 min read

Every spend classification vendor quotes an accuracy figure. Almost none of them define it, and the number is close to meaningless without the definition. Here is how to measure it yourself, on your own data, in about an hour.

Why vendor accuracy claims mean little

“95% accurate” usually means 95% of rows received a category. That is coverage, not accuracy — a tool that puts everything in one bucket scores 100%. The number you care about is what share of the categories assigned are the RIGHT ones, and that can only be judged against your data by someone who knows your spend.

The measurement, in an hour

Take a random sample of 100 classified rows. Random matters. Reviewing the top 100 by value measures the easy end; nearly all classification error lives in the long tail.
Judge each one right or wrong at Level 2. Not Level 3 — leaf-level disagreements are often taxonomy preference rather than error, and you will argue with yourself.
Count. Correct divided by 100 is your accuracy at a roughly ±10% margin of error, which is enough to decide whether a tool is usable.
Sort the errors into types. This is the part people skip and it is the most useful output.

Error types tell you what to fix

Missing information. The description was a part number. No tool recovers meaning that was never there — fix the data, not the tool.
Ambiguous boundary. Packaging classed as indirect when you treat it as direct. Fix the taxonomy definition, not the row.
Genuine misread. The description was clear and the category is wrong. This is the only one that is really the tool's fault.
Brand name. Categorised by trade name rather than commodity. Fixable with a correction that should then stick.

A tool with 85% accuracy where most errors are missing information is performing better than one at 92% where the errors are misreads.

What good looks like

On typical procurement data with usable descriptions, 85–95% at Level 2 is a realistic range. Below 80%, check whether the problem is the data before blaming the classifier. Above 95%, check that your sample was genuinely random.

Measure it again after corrections

The number that matters is not the first run. It is whether your corrections stick — re-run the same file and confirm the rows you fixed stay fixed, and that the taxonomy has not shifted underneath you. A tool that forgets corrections makes you pay the review cost every single time.