Not All Page Views Are Created Equal

If you’re any kind of television fan (and I’m not here to debate the merits of being one), at some point A.C. Nielsen and his data-gathering progeny have been a royal pain in your remote. The Nielsen ratings have the power to cruelly destroy a show, however brilliant, if the numbers don’t meet expectations.

The good news is that Nielsen//NetRatings, the Internet audience measurement service partly administered by Nielsen, doesn’t wield that kind of force. There are just too many Web sites out there. Finishing 100th among the finite number of prime-time TV shows can be a death sentence; finishing 100th in a given week’s NetRatings probably means little more than a shrug, and a sanguine shrug at that.

But as NetRatings ascends to a position of reliability, thanks to its blimp’s eye view of who’s-surfing-where on the Web, it’s time to check that zeppelin for holes. Put another way, Nielsen//NetRatings may have its value as a guideline, but only if taken with a couple of silos of salt.

Paging NBCOlympics.com

News item: according to NetRatings, for the week ending September 17th, NBCOlympics.com registered the most page views of any Olympic-related site. NetRatings vice president of analytical services Allen Weiner subsequently commented that “by far the top site in the U.S. for pure Olympics coverage is NBC’s Olympics site.”

So, here’s the problem. A Web page could have a single picture or a thousand words, or a single word or a thousand pictures. In the end, it is still a single page.

I went to the NBC Olympic site and clicked a link for a feature on Olympic trampolining. (Really, I did.) When the new window loaded, I realized I was at page one of the feature — page one of seven.

Each page had a sparkling photograph, but only one paragraph of text. This story could have been told in fewer pages, making me wonder if the design choice was a creative decision or a calculated one.

Quantity Isn’t Everything

NBC’s Olympic site also does not make it easy to return directly to a preferred page. No matter what page I was on, the address window atop my screen read www.nbcolympics.com. When I reopened NBC’s site to show a colleague another feature, it took me several clicks to find it, spending maybe five seconds per page. In short, NBC made a page view mint off me.

Not that I’m faulting NBC, which is getting enough Olympic-related criticism these days, for its Web layout. I’m just picking on Nielsen for suggesting that the quantity of page visits makes NBC the tops. Even from the standpoint of advertisers looking for the most efficient way to spend their dollars, quantity isn’t everything.

Is Time Money?

To its credit, NetRatings does provide supplementary information: the average time each visitor spent on the site. For the week ending September 10th, Microsoft (No. 4 for the week) had a unique audience of 16.8 million; eBay (No. 9) had a unique audience of 5.2 million. However, the average eBay visitor spent 52 minutes on the site, while the average Microsoft visitor spent less than six minutes.

Whose site deserves your esteem — or more relevantly to e-commerce, your marketing dollars? I’m not an advertiser, much less a statistician, but I assume some formula merging the unique audience and time data must be necessary.

Still, before you decide the math is as simple as that, ponder this: on the Internet today, not even minutes are created equal.

Theory of Relativity

In the TV world, 60 seconds is 60 seconds. Commercials run at the same speed, whether you’re watching a 48-inch Philips flat screen hanging on your wall or the five-inch black-and-white you’ve had on your kitchen table since 1975.

In the computer world, with its wide range of available hardware, it’s anyone’s guess if a visitor to your site is looking at the finalized page, content and ads intact, or whether they’re just watching everything load, load, load. Little known fact: the NetRatings clock starts the second a Web page is clicked, before it has finished loading.

If you’re an advertiser trying to reach me, you’d better catch me at work with the DSL connection, rather than at home on my Powerbook 3400 with its semi-reliable 28.8 bps modem. If you’re a Web site operator, you might just want the opposite, to build up your Web stats. Either way, Net Ratings doesn’t adjust for the hardware distinction.

Samples, for Example

Finally, consider the issue of statistical sampling. NetRatings says it collects real-time data for its U.S. rankings from more than 65,000 panel members. Somehow from this sample, NetRatings can tell us that 65.9 million people surfed the Net in the first full week of September.

I know, I know. I know I should believe that sampling works. In fact, I have more desire than the average person to believe it, because I once spent a rather interesting summer recruiting Washington, D.C. households to be Nielsen TV families. That’s right — at any moment, I was the one who could knock on your door and deputize you for airwave control.

The point is, I did get real insight into Nielsen’s workings, and came away impressed with the company’s methodology. Let me tell you, however, that the factors that determined who became Nielsen families were not perfectly random, nor perfectly objective.

Tallying the Numbers

Nielsen//NetRatings is not the only organization that counts page views — and the counts vary widely from source to source. According to a Wall Street Journal article this year, Salon.com said it recorded 3.7 million unique users in March, but ranking service Media Metrix said the hip content site registered only half that many.

Ultimately, many people will come to rely on NetRatings, for many good reasons. After all, NetRatings has no interest in producing flawed data, right? Nevertheless, everyone should keep in mind that the numbers don’t always add up.