Following on from my post the other day about Page Navigation to garner you some deeplinks. I also had a brainwave about how to get Google crawling even deeper into your posts.
First off, Whenever I build an automated site. That is either scraping content from elsewhere, using datafeeds, or sometimes even my very own content. I like to have the webpage be generated dynamically everytime. By that I mean, if you came to my homepage, and then refreshed, you would see two different web pages. Maybe not completely different, but maybe different stories, different links, different images etc. When I think about some of the most crawled pages in the world, they are the ones that are forever changing. Whether that be legitimate change or not. For example Twitter’s homepage is constantly changing (With all the tweets). Even with most news outlets. They have constantly changing homepages because the latest News is forever changing.
Google loves all of this.
It wants to keep up with the play of the world. And if you are constantly changing your homepage, then trust me, it will crawl you like mad.
So how to get the Deeplinks flowing, and Google crawling? The first step is to give it some random links on the sidebar. For me, it meant installing this plugin : http://www.romantika.name/v2/wordpress-plugin-random-posts-widget/.
It basically shows random posts on your sidebar in WordPress. In my case, I actually made the header “Popular Posts”. If you were using a site with tangible products, you could label it “Hot Products” or similar. All in all, It doesn’t matter in Google’s eyes what you name the widget. But when visitors come, if they see the words “Random” there is not really any reasoning for them to view it. Maybe if they were bored, but otherwise why do they care about some random post on your blog. However if it is named “Popular” or “Hot” then they think it is the very best of your posts, and will immediately check it out. I personally do this with most of the blogs I come across in the Affiliate Marketing Niche. I always head straight to their most popular posts.
Anyway, Back to work.
Pro Tip : It is also important to note that if you want to really make this change every refresh, you have to turn off all your caching plugins. Things like WP-Super-Cache, will cache your page, meaning that each load it will not change. It will simply be loaded from your WP cache.
Something I also ended up doing on one of my autoblogs was to implement the “Advanced Random Posts” plugin. I actually installed this one by accident, thinking it was going to add random posts to the sidebar like above. But it has turned out to be a diamond in the rough. This particular plugin will add X amount of random posts onto your actual homepage. You can place them after X amount of posts aswell. So essentially, Each time your homepage loads, there is a complete new set of posts on the homepage. This make’s Google think that you have all these new products/posts flowing in, and it will crawl like mad.
I do want to say though, that this really isn’t ideal if your blog is personal, or if you are looking to build up readership. These are more for autoblogs or automated data feed sites that are relying on one time visitors, not building an audience.
Just to give you a bit of inspiration, and proof that using these simple deep linking techniques (Hell you only have to install plugins!) really works. Here is a quick picture of Google’s crawl rate on one of my sites.
It is a fairly new site. Not even a couple of weeks old I think. Although I will admit, I have owned the domain for quite some time, but never really had much on there. As you can see, in a matter of days since I implemented all these changes, I went from pretty much 0 crawl rate, to almost 700 crawls a day (And still rising). It may not seem that magnificent to some, but this is really quite an amazing feat considering most people consider getting Google to just index your homepage in the first few days of a new site is good work. And once again, I will point out. There was no secret linking techniques used here. There was a few blog comments done (20 odd?), and I used Digg.com on one post and that is it 
Is it better to have those random posts have a relative nofollow ?
It shouldn’t make all too much difference. Generally speaking, I let PR flow freely around any one of my sites. And only nofollow outbound links. Even then, Google has set a couple of times, that nofollowing links doesn’t mean that the original page keeps the PR.
btw there is also one plugin that i find really good, its called Old Post Promoter, the name speaks for itself, check it out Wade