Alexa: Amazon’s Most Flawed Idea
What does a high Alexa rating mean to a web master? It shouldn't mean that much as it's not accurate. Alexa is a website that tracks a website's traffic history, and gives a ranking based upon the number of visitors. However the fact that it requires a tool bar to work flaws it in many ways.
Alexa works by counting on users to download a tool bar for Internet Explorer. It tracks what sites you go to, and compiles the data. For each time you visit a website using Alexa tool bar it multiples that by ten, because it is assumed that for every one Alexa tool bar user there are nine non users. This is also known as an estimate. On the tool bar it will list the rank of a website, and what other sites a user of that site tends to go to. The tool bar offers nothing else for the user except a search bar, which has terrible search results. It's also an eye sore for Internet Explorer 7.
Alexa has no support for FireFox, Opera or Safari at all. They refuse to make a tool bar for any other browser. But wait all is not lost - they offer solutions.
Build Your Own Alexa Toolbar. Because Firefox is an open source browser, an enterprising developer might make use of that functionality and our Web Information Services Feed to create such a Toolbar.
Yes, build your own. They want to compile user traffic data from people all around the world, and their advice is to build your own tool bar. Their other solution is to ask FireFox developers to build one into FireFox.
Ask FireFox to include Alexa. In 1998, Alexa built a feature for both Netscape and Mozilla browsers called What's Related. Unfortunately that feature is not included in the FireFox browser. Write a note (http://www.mozilla.org/contact/) to the folks at Mozilla/Firefox to let them know you want Alexa AND FireFox.
Apparently the folks at Alexa have never heard of any other browser besides Internet Explorer and FireFox. This seems quite unprofessional coming from a company owned by Amazon.com.
What's the problem with this you ask? Well the results are skewed - totally inaccurate. For instance if you look up the ranking for msn.com they are ranked #2. Now on a default installation of Windows XP, which is what the majority of people use; MSN is the homepage for Internet Explorer. All those IE users going to the MSN homepage gives MSN an unfair advantage. Take a site like Slashdot or Digg, with their user base being an incredible ~76% FireFox users. IE users on those sites are few and far between, and the Alexa tool bar and IE users are even more uncommon. Another website being hit as hard with these inaccurate statistics is MacRumors. Their site only has 13% IE users total, while FireFox and Safari are used about the same.
Aside from being flawed to begin with, there will always be people trying to exploit or game the system. AlexaSurf is a website designed to do just that. Essentially you visit other people's websites with Alexa's tool bar, and they do the same for you. Given enough time it could potentially increase your Alexa rating significantly.
With a growing amount of people using flawed and easily manipulated data in legitimate statistics there will always be someone to believe its accuracy. EBay auctions are using Alexa as proof they are selling a high traffic domain, scripts are being made to calculate website value based on Alexa ratings, and reputable news websites are using Alexa data to write articles about the biggest website in the world!
In short, the Alexa ranking system is naturally flawed. The data should never be treated as accurate, as it's easily manipulated, and not supported for most browsers in the world. It's an estimate, and nothing more.
Print Post
4:26 PM on October 19th, 2006
Well said and concisely documented. Alexa is a frustration more than a tool to me and many I know, and it’s nice to see others calling out the flaws.
5:19 PM on October 19th, 2006
As long as Firefox market share is under 50%, Alexa results may be a bit skewed but they will remain fairly accurate. “Most browsers in the world” are IE, actually, depending on how think about it. I know you mean browser apps, but Opera and Safari have close enough to zero market share to ignore them completely, unless you are selling Mac software.
5:56 PM on October 19th, 2006
So, Alexa is deeply flawed because it only samples from 91% of browsers? Other statistical systems should be so lucky. That’s quite a broad and even sampling of web users.
Yes, at Amazon’s size, they could - maybe should - just write a Firefox plugin. That said, you’re going to get a much, much larger statistical effect from the differentiation between people who do and people who do not install toolbars than from the relatively small browser divide.
Alexa is absolutely raking in the money. If you’ve got a better idea, get hacking; you could be rich. Until then, maybe consider that the thing you’re complaining about just isn’t that big a problem. All things equal, when you look through website metrics, with a few rare exceptions, from sector to sector browsing stats are almost uniform. The differentiation just isn’t that important.
6:32 PM on October 19th, 2006
The point about Firefox is well taken. But, where is the EVIDENCE that Alexa is flawed? In other words, for any random site, what are the chances that Alexa leads to biased info? Anecdotal evidence does not get us all the way there.
6:48 PM on October 19th, 2006
Plus, I notice Alexa results are HIGHLY skewed towards Silicon Valley web sites. The techies who bother to install Alexa tend to group around Silicon Valley startups (ie web 2.0) and push those sites up in the rankings. Hitwise just showed a chart of early adopters of YouTube, and notice how it is geographically skewed to the techies: weblogs.hi...ly_ad.html
7:12 PM on October 19th, 2006
Well I have already put my blog as the homepage for IE with Alexa toolbar installed…but if firefox & other browsers like Flock will have alexa toolbar then it will be great:)
7:29 PM on October 19th, 2006
Totally agree with you! thanks for lightning things up
1:10 AM on October 20th, 2006
To StoneCypher:
w3schools.com list IE usage at 60%. (http://www.w3schools.com/browsers/browsers_stats.asp)
e-janco.com lists it at 75% (http://www.e-janco.com/browser.htm)
Hitslink.com lists it at 82% (http://marketshare.hitslink.com/report.aspx?qprid=0)
safarla.com lists it at 55% (http://www.safalra.com/website/web-browser-market-share/)
tbray.org lists it at 35% (http://www.tbray.org/ongoing/When/200x/2004/12/12/-big/Browser-Market-Share.png)
websidestory lists it at 86%
theconter.com lists it at 84%
onestat.com at 85%
adtech at 83%
net applications at 82% (Those last 5 were the latest from each of the counting companies as listed at en.wikiped...b_browsers)
I got those numbers by going to the top websites by searching for “browser market share” on google. As you can see, they vary widely (from 35% to 86% - but none as high as 91% - as you state…) They vary for the same reasons that alexa is inaccurate - because they each are samples of very different users.
So, let’s take an average of those numbers - and say that market share of IE is 72%. That’s assuming that the fact that they were pulled from the first page of Google Search results that they have fairly even traffic between the different sites.
Now - let’s follow your argument about sampling 91% (though it’s really 72%). Alexa is not installed on EVERY computer running IE - just a subset. So Alexa is installed on some percentage of 72% of browsers. Not only that, but the browser a person uses is directly tied to what sites they vist.
It would be similar to taking a poll of public opinion of birth control - but limiting your set of people you can poll to Catholics. Since there is a direct correlation between being Catholic and opposing birth control, your results will be skewed.
You would be fine to select from the subset if a) there was no correlation at all between the selection pool and the measured effect, or b) you performed a weighted average of the results with results taken from the other subset.
And browsing stats are *not* uniform, as you mention. That’s the whole reason why this debate began in the first place.
2:05 AM on October 20th, 2006
For Alexa to be halfway accurate…it would have to take into account the types of browsers visiting a given site.
Their attempt to make the results real is a joke (guessing–wildly–that 1 in 10 users have Alexa). It may make results a little closer to what it really is, but it will also depend highly on the site they’re visiting. A good example given by the blogger is Mac Rumors…13% of it’s visitors are using IE. So 87% of people visiting the site have no chance of being counted…does this mean they should multiply Mac Rumor’s score an _additional_ factor of 7.5 (roughly 100/13)? Though this accounts to some people probably getting counted twice because we’ve already taken into account that 9 out of 10 people don’t have it. Though, you can swing it the other way and say that Mac Rumor’s IE browsers being counted by Alexa are worth even *more*, because maybe, one could say, they’re being visited by less than the 1:10 ratio of AlexaIE:nonAlexaIE, because obviously, a Mac Rumor’s visitor is too smart to install such nefarious software, making the Alexa browser far far less common on that site. *chuckle*
…my little rant…sorry :)
3:03 AM on October 20th, 2006
Actually there is an alternative for FF users via an extension called SearchStatus (and the link is provided off Alexa’s toolbar page):
quirk.biz/...rchstatus/
But I do agree with you that it can be gamed. However what other *free* alternatives are there?
Someone at Slashdot said that Google could do something similar, and seeing as how their toolbars are almost ubiquitous across browsers and platforms, the data would be more accurate. But they’re not. ;)
3:18 AM on October 20th, 2006
Alexa is a voluntary survey. It is as accurate as it’s sample space is random. If a web site that has a modest viewership were to successfully promote Alexa, their ranking would go up. It is interesting to see for any site that does not promote Alexa. It is otherwise no more accurate than those voluntary on-line polls. Firefox does have several extensions already that implement the Alexa stuff - just do a web search for “firefox extension alexa”.
5:37 AM on October 20th, 2006
The reason so many people believe that the Alexa data is useless is because of the different types of users that tend to choose a browser. Ine general IE is used by the lowest common denominator, novices and people who don’t know any better. These people are also the least likely to actually purchase anything online.
No matter what the market share of “alternative” browsers currently is, THESE users are what make up most Internet usage. It would be as if you took a survey regarding what kinda of cars everyone likes, but you only did this at a dealer, deciding to leave out anybody who works on their own car at home. Maaybe only 10% of the country changes their own oil, but that 10% is also the core of the automotive market. These are the people that influence their family and friends opinions. You would be committing corporate suicide by not including them in your survey.
Basically, that’s what Alexa is doing. You could use their data if you want to, but you’re basically ignoring your core market. As a business man it doesn’t make much sense to me…
9:29 AM on October 20th, 2006
“It is as accurate as it’s sample space is random.”
Ehm sorry, but that’s just not true, the users that have installed this toolbar are a very specific subset of all Internet users. The very fact that they even know about it and have installed it make them different than the other 90% of IE users (the 1 in 10 figure they named) and it makes them very different from the people that use non-IE browsers. You just can’t say with any measure of confidence that what holds true for Alexa-users holds true for others.
The people analysing opinion polls for example know that as well. If you go ask people in the street about a certain subject it all depends on where you ask, many polls are only held in big cities because it gives you easy access to a lot of people but city people or different from villagers. Even if you call a random selection of people from across the country the results can be skewed because of the fact that not everybody will be willing to answer your questions. For every person that tells you to bugger off and leave them alone your results become less usefull.
12:24 PM on October 20th, 2006
“So, Alexa is deeply flawed because it only samples from 91% of browsers? Other statistical systems should be so lucky. That’s quite a broad and even sampling of web users.”
No Alexa is not sampling 91% of users! Read the article and the other comments. Its sampling from the unknown, and probably not large, percentage of IE users who’ve bothered to install it. Add to that that its not exactly a random sample - its heavily skewed to the more technically proficient.
2:56 PM on October 20th, 2006
Also, Alexa doesn’t count HTTPS pages.
alexa.com/...learn_more
For a long time, LinkedIn’s Alexa numbers were essentially 0, because the whole product ran through HTTPS.
8:02 PM on October 20th, 2006
Nielsen..Netratings, Comscore Mediametrix, Hitwise all offer far more data than available through Alexa. They charge tens of thousands of dollars for it…and still have less than perfect data.
Alexa is great for a rough and ready ballpark. It’s just a litle scary if Alexa gets quoted as the high standard, ahead of the likes of the higher integrity paid services.
I list rankings of Online Personals sites on OnlinePersonalsWatch.com courtesy of the paid sites, and also include some rankings from Alexa.
Mark Brooks
editor
onlinepers...swatch.com
6:11 AM on October 21st, 2006
It’s quite ironic that this post boosted my Alexa rating over 35 thousand to 4.7 thousand for the day.
11:21 PM on October 21st, 2006
I host a weekly (actually aired, not simply a podcast) radio program called TechTAK. We’ve been going for 2 years now and just recently began doing product reviews. When corresponding with one of the vendors, they wanted (required, it seems) our Alexa rating. This is a nice little article with some decent info that I’ll be passing along to vendors who seem to think that what most of us consider spyware is something we should dtrive to be placed in a good rating with…
2:31 AM on October 22nd, 2006
Any survey whose users are self-selected is inherently flawed. This includes the surveys on most web sites (which are really just a scam to drive page views), as well as schemes like Alexa.
8:39 AM on October 30th, 2006
Alexa rankings are very flawed and we have just launched a website to prove it:
alexasucks.com
Check it out and lend a hand!
4:05 PM on December 1st, 2006
Anyone else having trouble with Amazon deliveries?
I am real late on a delivery of a book from the USPS, sent a good full week ago…was “estimated delivery” to be here last Thursday.
I'm worried they may have lost it.
Amazon did their job and had it out the morning after my order, but the clueless USPS…?
Anyone one else have trouble?
Thanks for the feedback..
12:17 PM on August 21st, 2007
[quote comment="11090"][...] While I don’t necessarily like Alexa, I like what they are trying to do. Their results aren’t fully accurate, but it’s really one of the best resources we have. [...][/quote]
Recources for WHAT? Why do we need a resource for spyware? People need to wake up and stop being so sheep-like. Geez. “Let’s all let some company put spyware on our computers because it’s the only resource we have to make sure that those websites have visits by people foolish enough to install a spyware toolbar on their computers…”