A useful, underused signal in evaluating AI technology claims is watching for metrics a vendor used to prominently disclose and has since quietly stopped mentioning. The silence itself is informative in a way that's easy to miss if you're only reading current marketing material rather than tracking it over time.
Why vendors stop disclosing a metric
A metric generally gets dropped from marketing material for one of two reasons: it stopped being favorable, or a competitor's equivalent metric became more flattering by comparison, making continued disclosure a liability rather than an asset. Neither reason gets stated explicitly - the metric simply disappears from the next round of materials.
How to actually catch this
This requires comparing a vendor's current claims against their own past materials - press releases, product pages, conference presentations - from a year or two earlier, which is more work than reading current marketing alone but is exactly the kind of longitudinal tracking that reveals a pattern a snapshot can't.
What a dropped metric might indicate
A vendor that once touted a specific accuracy rate, response latency, or benchmark score and no longer mentions it may still be delivering on that dimension quietly, or may have quietly regressed as they optimized for something else, or may simply have shifted focus to a genuinely more important metric. The dropped disclosure doesn't tell you which explanation applies - but it does tell you the question is worth asking directly.
Why this diligence step is worth the effort
Directly asking a vendor why a previously highlighted metric no longer appears in their materials, and specifically what the current number is, is a pointed question that a vendor with a genuinely good current answer will typically answer. Reluctance to answer directly is itself useful information.