Dynamic, database-driven websites are great. They allow business owners the ability to use a content management system (like WordPress, Joomla, or Drupal) or an ecommerce shopping cart platform to add hundreds of products or pages without having to create individual html pages. I have created hundreds of such websites for my customers over the years, and normally, Google can find these sites without a problem and view their content. But sometimes it can’t. For no particular reason, Google simple seems unable to index some sites, no matter how many SEO tactics you employ – including having hundreds of in-bound links to your site.  If you go to Google and type in “MySite.com”, you would certainly expect Google to bring up “Mysite.com” at or near the top of the search results. Sometimes it doesn’t and you have to ask: why? I have found that problem appears to happen most frequently when moving from a static html site with lots of indexed pages by Google to a database driven website where none of the previous pages exist. Instead your url’s now contain a question mark with variable parameters such as mysite.com?id=2.

Here is the “solution” I have come up with. (Note: If this is too technical for you, employ your webmaster to do this for you).

First, go to Google webmaster tools:

Sign up for an account and login.

Add your site. Upload the file provided by Google and verify that you own this site.

Now that your site is identified by Google, go to the menu on the left and select
Diagnostics ->Fetch as GoogleBot
This will allow you to see whether or not Google is able to crawl your site.

 

If you get an unreachable error, Google is unable to find your pages. This is a problem and you MUST correct it. Even if you get a success message, when you have a database-driven website, Google may only be finding a single page because it thinks that index.php or index.cfm is a single page and is not reading variable data behind the ?.

Go to your website’s home page, right click on the content and select “View Page Source”. Save this file as a simple htm file, such as “home.htm”. Alternatively (and perhaps better), is to manually create a simple page with all your important links and save this as a simple htm file. Upload this simple htm file back to your website, and test it.

Now find a free xml sitemap building tool such as this:

Many of the sitemap building tools cannot crawl dynamic database-driven websites any better than Google can. The one I’ve listed above can. Generate the site map by supplying both your domain name and the file you just created, such as:

www.MySite.com/home.htm

After the sitemap has been generated, save it and copy it up to your website. You should place this file in the root folder of your site.

Now go back to Google webmaster tools. Select “Site Configuration – Site Maps” and click on the button to submit a site map.

Select “Diagnostics -> Fetch as GoogleBot” again. This time, enter in the name of the htm file you created, not just your domain name. For example: www.MySite.com/home.htm. You should now get a “Success” message!

Click on “Submit to index”. Pick on the radio button to submit both the URL and all linked pages.

Give Google about 24 hours to include you in their index. From now on, Google should be able to find your site (and all of the pages that link from your sitemap)!