Improving organic Google search results for your business’ WordPress-based website depends on ensuring that Google knows about every page, every post, every media item. To inform Google of your site’s content, we use “site maps” which are files that contain links to every aspect of your website. These sitemaps are submitted to Google so that it knows where to send its web “robots”.
Generating sitemap files can be quite involved if you attempt to do it manually, particularly if you have a lot of content on your site, such as products or a long history of posts. They are generally created in XML format, which
To overcome the challenge of manually creating sitemap files, plugins are available that will generate them for you. This can be an incredible time saver, though they are not without their own risks.
Installing a sitemap generator is no different than most WordPress plugins. Identify the one you wish to use, install it. Answer a few questions in the provided settings interface, such as whether to include or exclude static pages, posts, media, etc. You should also have the option of indicating how frequently your pages & posts are updated (hourly, daily, weekly, monthly, yearly).
Submitting the index file to Google is also quite straightforward providing you’re familiar with validating your website with the Google Webmasters service. You may need to place a simple text file on your website or add a small entry to your DNS records to prove to Google that you are an authorized agent of the website.
Once your sitemap has been submitted to Google Webmasters, Google will test the links and crawl your site over time with its Googlebot spider service. Googlebot will capture your website content and determine how useful it will be to the general Internet population in their searches. The more relevant your content is, the higher up in the organic search results your site will appear.
When all of the moving parts are functioning perfectly, it can be great for your search results and your business. But how do you know that everything is working correctly?
Google Webmasters will inform you of issues with your site. It will indicate errors with how it crawled your content and provide some relatively cryptic explanation of what needs to be repaired. It will show you what pages it has indexed, how frequently it returns to detect changes and any recommendations for improvement.
Listen to Google Webmasters, particularly in regards to sitemaps. This morning we received an email from one of our valued past customers who received notice from Google:
“Hey guys… I just got an email from Google saying that a new issue was found, ‘Submitted URL marked noindex.’ I promise we haven’t changed anything, just staying on top of our updates like you said we should.”
Therein lies the problem: something changed not in the site’s content, but likely within the plugin that was managing their sitemaps. While installing updates on a regular basis is a good practice, they are still at risk of programming bugs, conflicts and unintended consequences of changes.
While digging through the Webmaster results and testing the sitemaps, we discovered that a recent update to the plugin added ‘noindex’ to the site’s dynamically generated robots.txt file by default! Gasp! This is essentially telling Google, “Here are all of my links….just go ahead and ignore them.”
It appeared that many users of the plugin were encountering the same issue. This is a widely used plugin, too. We also discovered that deeper links were now incorrectly formatted in XML and were considered invalid by the hundreds.
Quick response to remove the offending plugin, identify and test a few replacement options and resubmitting the rebuilt index file should repair any damage. But it required prompt action and a deep familiarity with how search engines work to mitigate any damage.
Lesson: stay on top of your updates and review every message from Google Webmasters the moment it arrives. Search for a fix, but if you tread into unfamiliar territory, don’t hesitate to contact your site administrator… or us. We’re happy to help.
