Posted by randfish
Howdy gang! As promised, last night we launched our 45th Linkscape index. You’ll find new data in Open Site Explorer, the Mozbar and the PRO Web App as well as in our API. We’ve also started to address some of the challenges discussed in prior Linkscape data, which I’ll cover below.
Here are the metrics for this month’s update:
- 44,210,612,409 (44.2 billion) URLs
- 452,126,131 (452 million) Subdomains
- 104,185,923 (104 million) Root Domains
- 360,491,328,983 (360 billion) Links
- Followed vs. Nofollowed
- 2.21% of all links found were nofollowed
- 58.95% of nofollowed links are internal, 41.05% are external
- Rel Canonical – 10.12% of all pages now employ a rel=canonical tag
- The average page has 77.47 links on it (down from 80.08 last index)
- 65.23 internal links on average
- 12.24 external links on average
As I noted in the September index update, we have had some serious issues when crawling deeper on large domains and encountering binary files that contain code our crawler recognizes and treats as a link. To help stop this problem, we applied a black list to this index to stop a large number of the files folks had reported to us (our estimate is that ~40% of binary files are now removed). However, we know there’s still more than a few of these in the database of links so we’ll continue cranking away on solutions to remove them all. Our hope is to have them reduced in the next index (November) and nearly eliminated by the December index. If you’re ever curious about the next/previous updates, you can always see data for them on our Linkscape calendar.
I’m excited to announce that we’re also just a couple months away from showing historical Linkscape metrics data in the web app. In the next 60(ish) days, we’ll be launching a tab in the Link Analysis section showing topline link metric history for your campaign’s site and its competitors. There’s also tons more good stuff coming to the App before year’s end, but I’ll save those announcements for other posts.
But, perhaps the biggest win with this index is the full functionality now available through the domain "drilldown" feature in OSE:
You can now click on any domain in the "linking domains" view to see a list of all the URLs we found from that particular site pointing to the page/domain in question. It’s a UX upgrade that, IMO, completes the clean, usable experience inside OSE and provides a view that marketers consistently want to see. Many thanks to the Linkscape + OSE teams for getting that included.
As always, if you’ve got feedback about our link data or the latest index, please leave a comment. Our engineers take suggestions very seriously. Thanks much!
p.s. I’d incorrectly labeled this as the "November" update, when it’s obviously still the middle of October… Doh! Fixed in title, but URL will be a reminder of my not-so-smart move.
