Ive been using wordpress for quite a while now, and can generally finish up a full installation with all the plugins i need within about 30 minutes. But lately ive been stung twice by , what i think is, a new setting in wordpress. Or atleast a new Default setting.
Using this blog as an example, i just moved hosts last week. For that i thought i would just do a fresh install, and then reload what i needed to. As i had a crap load of messed up files and it would just be easier to start everything new. For this i generally use fantastico. I can install manually, but fantastico is just that much easier and i havnt found any issues with it in upgrading or anything like that.
However, yesterday i checked my google webmaster tools. If you dont know what this is, essentially its just for webmasters to check on googles outlook of your website. So how often they crawl your site, your top search rankings, links to your site etc.
Anyway i looked at some of the numbers and noticed this :
Now even without me telling you what this graph represents, you can probably see what happened right at the end there. One massive huge drop to zero 
. This graph actually represents how many pages google is crawling per day, so you can see i had a big rise at the end of july, which is about when i started developing most of my products. And as you can also see, for some reason lately i have been given a big fat zero.
I checked some of the other tools in googles tools and found that for some reason, my “robots.txt” was blocking google. Personally i let wordpress generate its own robots.txt as i never have any set rules for it. The default is usually fine. Well, not anymore it isnt.
You see if you have a wordpress blog (And its on the latest version), if you go on the left hand side of your admin panel, click settings, then privacy, you will find something that boggles the mind.
Click To Make Bigger!
Now, i have changed mine. But the default is actually the “I would like to block” option. Maybe its because im looking at it from my end of the scale, but i cant imagine many people wanting to block spiders. Even beginner bloggers talking about their pet ferrets want to have a crack at the big G. And even if there is people out there that dont want google on their site, majority should rule (Or atleast sanity should), and the vast majority of people will have absolutely no reason to block google.
Ive been stung with this twice now. The other time was with a brand spanking new blog, that i was working so hard to get indexed and just couldnt understand why nothing was working. So if your not getting the google love that you expect, or your starting a brand new blog, dont forget to go change this option!
