Affordable Website Design Minneapolis Contact

Why Isn'y My Website in the Search Engines?

Another of the most common complaints I see on Google Webmaster Help Forum is "I can't find my site in Google!" Submission alone is not sufficient to get a website indexed. It will get your website's main page crawled, but that's just a first step. It's good to remember that the search engines are eager to include new websites in their index, so there is usually a simple explaination why a new site hasn't been indexed. This Search Engine Optimization Tip will show you the most common problems and their solutions.


Reasons Why A Website Isn't Indexed

  • No Inbound Links To The Site - You need at least one good quality link from another website to be included (ie. "indexed") in the major search engines. This is the main reason why new websites are not indexed. Start by submitting your site to a good quality directory like The Open Directory Project, JoeAnt, or one of the others I recommend in my Building Links SEO Tips article. Ask other websites whose main topic is related to yours to link to your site. A good quality link comes from a prominent page on a high quality website. Forum signatures and blog comments are not likely to help your ranking much, but they can help get your site indexed.

  • Site Blocked By The robots.txt File - The robots.txt file is a plain text file that controls how search engines access your site. Many website designers block the search engines from crawling a site while it's under construction, and then forget to remove that blocking instruction when the design in completed. So check your robots.txt file in Google's Webmaster Tools, where a tool is provided that will test your robots.txt file. You'll also find information there on how to use your robots.txt file properly.

  • Site Blocked By robots <META> Tag - Similarly, some web designers will insert a robots <META> tag set to "noindex" on a website's main page while it's in the testing stage. The "noindex" robots <meta> tag will prevent a page from being indexed, or will remove it if it has previously been indexed. Remove the tag and you should recover in a short time.

  • Server And DNS Problems - It's rare, but not unheard of, for server problems to be severe enough to prevent the search engines from accessing a site. Sometimes hosting services have automatic controls that see search engine crawlers (a.k.a. "robots", a.k.a. "spiders") as malicious and block them. Use the "Fetch As Googlebot" tool in Google's Webmaster Tools console to make sure that Googlebot (Google's crawler) can access your site. Extremely slow response times can also prevent indexing. An improperly configured Domain Name Server (DNS) can sometimes cause problems as well. Check your domain's DNS at intoDNS, and if their system reports problems, consult your hosting service about it immediately. If you don't get a satisfactory response, you should move to a new hosting service.

  • Frame-Forwarded Domain - Some free and low-cost services have webmasters create a <frameset> page as the main page of a domain name package, but the website's actual content resides on a completely different domain. The major search engines see such a website as having no content of its own and will not index it. In this situation, the search engines (epsecially Google) will often index the content at the domain where it truly resides. The solution is to buy a conventional hosting service, or at least use a reliable free host, so that your content is seen to be on your own domain.

  • Invalid/Poorly Constructed Webpages - The search engines' crawlers are very tolerant of errors in the HTML mark-up of webpages, but it is possible for mark-up and syntax errors to prevent indexing in extreme cases. Valid HTML mark-up will not only help insure that your site is indexed properly, it will also insure that your users see what you wanted them to see when they visit your site. Check your site's HTML with the W3C's HTML Validator and correct any errors it reports.

  • Previously-Owned Domain Banned/Penalized - If your domain name was previously owned and was banned for violating the Guidelines in the past, the search engines might continue to exclude it from the index. You can check the history of the domain name by using the Wayback Machine operated by archive.org. Look at the old pages shown in the Wayback Machine for signs of a poor quality website. If the website used to be about pornography, drugs, or other often-abused topic, or if the site looks like it was created by copying the content of other websites, there's a good chance it was banned. The solution is to file a Reconsideration Request in Google's Webmaster Tools console, or use the Email Support form in Bing's Webmaster Center. Tell them that you are the new owner of a domain you believe has been penalized, and that you intend to adhere to their guidelines.

As I say, the search engines are anxious to include your website so they can show it in the search results. You just have to make sure that they see signals of quality (like an inbound link!), and that your site behaves properly. Of course, being indexed is not the same as ranking well, but it's a very necessary first step in your Search Engine Optimization and Marketing program.


This page was last updated on June 13, 2015


Need More Help?
You'll find more SEO Tips on the menu on the right side of this page.
You can also contact me with your SEO questions.

If you can't fix your website search engine problems on your own, my Search Engine Optimization Services can give your website what it needs to get your business' fair share of search engine traffic quickly, without disturbing your website's design, and without breaking your budget.

Call Richard L. Trethewey at Rainbo Design in Minneapolis today at 612-408-4057 from 9:00 AM to 5:00 PM Central time
to get started on an affordable website design package or search engine optimization program for your small business today!

In writing these SEO tips, I'm often reminded of a pearl of wisdom that my high school computer programming teacher passed on from one of his teachers, "Computers are high-speed idiots!" Remember that, and don't let them get under your skin.


Looking for more website design information? See my Sitemap!

  Share This Page!  
Still Have Questions? 
Talk To Me! Click Here!