All of these third-party analytic services are terribly inaccurate but will continue to be used until a more accurate tool is available. Below is an overview of the methodologies used for collecting data for each of these services:
Methodology: Captures traffic data from users who download and use the Alexa Toolbar in their web browser. They also claim to use other “diverse traffic data sources”, which is very vague.
Sample Size: There is no known number of current Alexa Toolbar users, but it is said to be in the “millions.” I would bet that size continues to decline as more and more spyware programs flag the Alexa Toolbar as spyware/adware and either uninstall or completely block installation of the toolbar in the first place.
Problems: Well, there are a few:
1. Easily skewed – Since the Alexa Ranking is determined by users who have the Alexa toolbar installed, it is often in the interests of a website owner to install the Alexa toolbar themselves to increase their own Alexa ranking. And that is precisely the case. That is why you will see most webmaster forums and SEO blogs have high Alexa rankings, even if they don’t get much traffic overall.
2. Easily manipulated – Even though Alexa claims they have since added measures to prevent intentional manipulation of their rankings, it is still possibly to increase your own Alexa ranking by installing the toolbar and visiting your own site. Try it out for 7 days straight and I guarantee your Alexa ranking will increase significantly.
3. Catch 22 – Most webmasters nowadays are aware that the Alexa ranking is often inaccurate and thus say “oh the Alexa ranking doesn’t mean anything.” But in reality, most of them do care about it because many advertisers use Alexa ranking data to determine which websites they want to advertise on. And people looking to buy a domain name or website will often do the same in determining the value of a site. So it’s a Catch 22 – the webmaster knows the data is inaccurate and shouldn’t care about it, but they do care about it because advertisers care about it. And that will continue to be the case until more accurate data is available.
Methodology: Captures traffic data from “diverse sources” such as the Compete Toolbar and ISP logs.
Sample Size: Approximately 2,000,000 U.S. users, or 1% of total market
Problems: While not as easily skewed or manipulable as Alexa, the sample size is so small that the data is often wildly inaccurate.
Methodology: Quantcast measures websites that have installed their tracking code directly in the HTML of the website
Sample Size: They claim to have 80,000 “Quantified Publishers” who have installed the tracking code across 10 million websites.
Problems: Again, the sample size. The majority of websites do not have the QuantCast tag installed on their site so it is impossible to compare the traffic of a website that is able to be directly tracked with their tracking code and ones that are not.
ComScore, Nielsen, HitWise (I am lumping these three together for the time being)
Methodology : Most of these get their data from willing participants. Their users agree to install survey software on their computers to track their usage. These companies also get data from ISPs.
Sample Size: 2 million to 10 million in the US
Problems : Sample size, sample size, sample size!
(I will continue this post later, I just had to get something written since I’ve been meaning to for quite some time!)