You can always prove your point by defining what to measure. Jakob Nielsen is a renowned pundit, the leading evangelist for measuring results of user interface design and one of the most experienced and influential voices in design. His October 9th Alertbox shows how even the best pundits can fall into the trap of drawing conclusions using measurements from incompatible systems.
Nielsen’s intent is laudable. He contends that the vast majority of users are consumers of content instead of creators and is urging site operators to make participation easier.
User participation often more or less follows a 90-9-1 rule:

* 90% of users are lurkers (i.e., read or observe, but don’t contribute).

* 9% of users contribute from time to time, but other priorities dominate their time.

* 1% of users participate a lot and account for most contributions: it can seem as if they don’t have lives because they often post just minutes after whatever event they’re commenting on occurs.
Nielsen goes afoul when he talks about blogs, “There are about 1.1 billion Internet users, yet only 55 million users (5%) have weblogs according to Technorati. Worse, there are only 1.6 million postings per day; because some people post multiple times per day, only 0.1% of users post daily.”
“There are about 1.1 billion Internet users, yet only 55 million users (5%) have weblogs according to Technorati. This analysis has three fundamental flaws.
1. Nielsen takes the broadest measurement of internet users and the narrowest definition of a blog, posts using one of the popular blog publishing platforms such as WordPress that happen to report postings to Technorati. The total number of internet users is not the same as the number of blog readers. To assume that even half of that 1.1 billion “users” would recognize a blog as being a blog defies credibility so it makes no sense to count them when measuring participation. Does someone who has never been to a site count as a lurker or passive participant?
2. Comparing measurements of different values from two different systems is unreliable. It is perfectly reasonable to compare the measurement of the same value from two different measurement systems to compare the measurement approach. You can’t do analysis with measurements tfrom two different system without normalizing the data.

• Technorati is an American-centric company that doesn’t claim to have an even global penetration.
• Blogging is a new phenomenon that has only been in the public eye since 2003. According to Technorati sources, the number of blogs doubles every six months. The August figure of 55 million will be around 110 million by February of 2007. It is approaching 70 or 75 million as of the middle of October.

3. The biggest problem is the definition of “blog”? Blogging is a term that embraces much more than the enabling technology. It isn’t about posting your opinion in Word Press; it is about adding your thoughts, opinions and analysis into the public discussion. Nielsen’s AlertBox, for example, is an opt-in electronic newsletter that is also posted to his site. The content he writes is widely discussed in blogs and gets its share of links from the blogosphere. The only reason AlertBox is not a “blog” is that the technology he uses to publish. It shouldn’t surprise anyone that mixing a broad definition of users and a limited definition of participants produces lopsided results.
Worse, there are only 1.6 million postings per day; because some people post multiple times per day, only 0.1% of users post daily.” Nielsen disappoints again on this point in three ways.
4. Nielsen defines daily participation as the threshold for determining whether someone is a regular contributor to the discussion. His definition stipulates that quantity of posts is the only thing worth analysis. Take Alertbox as an example again. Since 1995, Nielsen has published his column roughly twice a week or about 260 times over 11 years. By his definition of participation, one of the most widely read column on User Interface and Design, written by one of the leading experts on the topic, counts as an occasional contributor.
5. He ignores comments as a form of participation in the blogosphere. Most blogs have at least a few comments and many have hundreds or even thousands. It is hard to justify not including these as participation.
6. The last issue on this point is how he crunches the numbers. Technorati’s measurements are “ancient” as judged by the historical rate of change. Still, if we accept 55 million blogs and 1.6 million posts/day, 2.9% of bloggers post EVERY DAY. If we use a more reasonable threshold like posting once a week or even once a month, participation in the blogosphere may be as high as 10 or 15%.
The 90-9-1 analysis ignores that fact that users are in multiple communities. A user who is a devoted participant in the Amazon community for example, one who has read thousands of books and posted reviews for each, can hardly be expected to be a daily blogger as well, but do you count him as a lurker?
You can prove almost any point with statistics and measurements tailored to your definition. If you want data for reliable decision making, you need to be more thoughtful in your approach.

Technorati Profile