I have a habit I picked up after watching a developer cite a number in three different posts that turned out to come from the same blog post, which itself came from a sample of eleven websites. The number was real. The conclusion attached to it was not. That is the moment I started reading the methodology on every web technology statistic before I quoted it, and it is the reason I keep a short list of datasets I trust by name. None of this is hard. It just takes thirty seconds per number, and it stops you from embarrassing yourself in public.
The trap is the sample, not the number
Every headline web statistic comes from a sample, and the sample is the part most posts do not name. W3Techs crawls live pages and reports what ships to anonymous visitors. State of CSS asks opted-in developers what they like to ship. Web Almanac measures pages from a different crawl that focuses on HTTP Archive data. Three different datasets, three different universes, three different answers to the same question.
If you read only the headline, you cannot tell which universe a number came from. A claim that one CSS framework rules the deployed web and a claim that another framework rules the developer mindshare can both be true at the same time, because they are sampling different populations. The trap is not that either dataset is wrong. The trap is that the headline collapses the difference.
A clean example is the gap between what crawlers see and what surveys report. The crawler numbers say Bootstrap dominates the deployed web, by a wide margin. The survey numbers say Tailwind dominates what developers say they want to ship. Neither is lying. They are sampling different universes. If you pick a framework because everyone uses it, you have to pick based on the universe you care about. That is a sentence I wish more framework blog posts would print at the top.
Three datasets worth knowing by name
I keep a short list of datasets I trust enough to quote. None of them is perfect. All of them tell me something true about a specific slice of the web. Knowing which slice is the difference between using them well and using them badly.
- W3Techs crawls live pages. Use it for questions about what is actually deployed on the open web. Take its framework numbers with a small grain of salt, because some frameworks are easier to detect than others from a crawl. The crawl misses single-page apps that lazy-load their CSS, which is a real gap.
- HTTP Archive / Web Almanac crawls a different sample, focuses on size and performance, and is the source I trust for median bundle weight questions. The Web Almanac publishes a chapter on CSS every year and that chapter is the closest thing to a public benchmark the wider community has.
- State of CSS / State of JS surveys opted-in developers. Use it for questions about what developers want to ship, not for what they actually ship. Treat the adoption numbers as floor, not ceiling. The opt-in bias (people who care enough about CSS to fill out a survey) skews the answers in ways the report does not always call out.
Three datasets is enough. Beyond three, the methodology differences multiply and you spend more time reconciling the numbers than using them. Pick the one that matches your question. Quote the exact sample size and the crawl date. Move on.
What to check before you trust a number
Every number I quote passes through the same four checks before I use it. None of them is fancy. All of them are things the headline usually does not tell you.
- The exact date the crawl ran. Adoption shifts year over year. A number from two years ago is not a current number, and a blog post that updates its statistics quarterly is worth more than a number frozen in a screenshot.
- The sample size and what counts as a sample. Eleven websites is not a sample. Eleven thousand crawled pages is. Eleven thousand opted-in developer respondents is a different sample entirely.
- The detection method. Some frameworks are easy to detect from a crawl (their signatures are in markup or filename). Some are hard to detect (their classes are utility-named and can look like hand-written CSS). Detection bias matters more than most posts admit.
- The denominator. A claim that a framework runs on 13.6% of sites means something different from a claim that it runs on 74.8% of framework-using sites. The first is a fraction of the whole. The second is a fraction of a slice. Both are technically true and both are honest headlines for two different stories.
Two of these are usually enough to settle the question. If the date is missing and the denominator is missing, you are looking at a number that someone has stripped down for punchiness, and you should treat it accordingly.
When the methodology check tells you to walk away
The check tells you to walk away when the number is being used to make a decision that actually costs you money or time. Picking a framework based on a tweet. Quoting adoption to a stakeholder. Writing a policy based on what someone said most of the web does. None of those decisions should rest on a number whose sample size, crawl date, and detection method you have not read.
For everything else, a headline number is a fine starting point. The check exists for the moments when the number is doing real work. That is when the methodology matters more than the headline.
Trade-offs
The check is not free. It takes thirty seconds per number, and most articles cite more than one number. That is a real cost on a busy day, and it is one I accept for any number I plan to act on. The check also biases me toward datasets that publish their methodology in detail, which means I miss some genuinely good signal that comes from smaller or less rigorous sources. That is a real loss, and it is the one I weigh against the cost of acting on a number I did not read.
The check does not generalize to AI-generated statistics, which by design hide their sources. For those, the check stops at the date-and-denominator test and the rest is a leap of faith I would not recommend. It also does not generalize to statistics about user behavior (clicks, time on page, conversions) where the methodology is often locked behind a vendor portal. For those, the only honest move is to ask the vendor.
Bottom line
Before you quote a web technology statistic, run the four checks. The exact crawl date, the sample size, the detection method, and the denominator. Two of them are usually enough to settle whether the number is doing honest work. If the number is doing honest work, quote it with the dataset name and the date attached. If it is not, walk past it. The methodology is the only part of any web statistic that does not go stale. Read it first, quote it second, and you will embarrass yourself less often than the people who do it the other way around.