I built a pipeline to fact-check startup claims. My own data killed the marketing stat I wanted to publish. [I will not promote]

I build screening software, so I wanted a scary number for marketing: "X% of startups inflate their claims." I ran claim extraction and verification on 11 live equity raises (StartEngine, US). 56 checkable claims total: named clients, certifications, valuations, raise amounts, user counts. First pass looked great for the narrative. 73% of companies had at least one claim that didn't hold up. 36% had at least one flatly contradicted by external sources. Then I looked at the contradicted ones. Every single one was a point-in-time number: valuation, total raised, user count. The pipeline was comparing the number in the current raise against third-party agregators that were a round behind. A valuation that moved between rounds is not a lie. It's two snapshots taken at different times. So I did split claims into two buckets: Durable claims (named clients, partnerships, certifications, founding facts): Out of 25, 19 verified, 5 partially supported, 1 contradicted. The one real catch: a company listing four blue-chip clients, one of which no source could confirm. 2. Point-in-time claims : This is where all the "contradictions" lived, and most were staleness artifacts, not deception. The honest conclusion : on this sample, the "founders inflate everything" story doesn't hold. What actually broke was the data layer everyone screens against. Valuations, raise totals and user counts are unstandardized, updated on different clocks, and nobody reconciles them. If your screening compares a deck against an aggregator, a lot of your red flags are just clock skew. What changed in how I screen : Date-stamp every number before calling anything a discrepancy. For US raises, the SEC filing (Form C) is the source of truth for amounts, not aggregators. Separate "who they say they work with" from "how big they say they are." The first is checkable and occasionally false. The second is mostly just stale. To be clear, the takeaway is not "verification is pointless." The takeaway is that naive verification is worse than none: it cries wolf on stale numbers and buries the one real catch. Doing it right means dating every figure, picking a source of truth, and separating durable claims from snapshots. That's tedious at 10 dossiers and impossible at 200 by hand. This is n=11, so not a stat, and that's part of the point. I almost published the 73% number. It would have been exactly the kind of inflated claim the tool exists to catch. Curious how people here handle this at screening stage: do you date-stamp figures from decks, and what do you treat as the source of truth when aggregators disagree?

添加评论
点赞收藏
点踩分享查看原文
评论
?
参与讨论