Everybody wants their site to be indexed on top by the search engines.
Indexing or not indexing the becomes a prime concern once a site is submitted to the search engines. Due to several reasons as well as due to some common mistakes, a site is often not indexed by Major Search Engines.
Let’s look at some of the probabilities for which a website may not be indexed.
Index Time: It often requires one to eight weeks for a site to be indexed. So enough time is to be given to let a site get indexed by the major search engines. You might come across the amount of time required to index your website from the search engine’s submission page.
Limits for Submission: Many of the search engines maintain a limit for submission for a domain. You have to be aware of that particular limit of submitting pages and make sure you have not exceeded that limit. Many are of the opinion that not more than a page should be submitted per day for a site.
Deep Links: If only the home page of a site is submitted to the search engine, then there is a chance that the crawlers will visit only one or two links from the submitted page. There is a high possibility that the bots will not go deeper to the site. That is why it is better to submit the pages individually or you have to create more number of direct links from the site.
Missing Page: Often a common mistake takes place while submitting the site to the search engines. It is often seen that the pages are not uploaded to a site before submission. It is utmost important to check that all the pages exists before the site is submitted.
Placing External Links: Some search engines like HotBot and Google do not index a site that does not have links to any other sites. The search engines might index only the home page of such sites and there is high probability that the other pages of the site would not be indexed.
Link from Homepage: Sometimes search engines refuse to index those internal pages of a site that are not linked from the home page of that site. And the search engines tend to drop those pages. If the crawlers do not find a roadway to reach your internal pages from the index page then they might consider them to be less important.
Frames: It is better to create non-framed versions of your sites. If you put content inside frames, then problem may arise with their submissions to the search engines. For framed pages, search engine may read the main content but not the surrounding menu frame. In that case only a part of the content would be visible to the readers.
IP Address: At times a site can be ignored from not being indexed as that particular website seems to share the same IP address with other sites. So it is advisable to use unique IP for each sites and prevent them from being banned through repetitive uses.
Dynamic Pages: Many of the search engines avoid indexing the dynamic web pages with special characters like an ampersand(&) and question mark(?). The search engines tend to leave the portion with Java Script and index the rest of the page. So a static version for each of the pages you want to be indexed, should be created in this case.
Technical Error: Random errors can take place in the search engines due to bugs and technical problems. Search engines run database of innumerable pages and error can take place during submission of web pages. Many prefer re submitting their pages every month so that none of the pages are missed out.
Spam: If a site is full of spam content then a search engine would not prefer to index its pages. Excessive use of keywords or attempts to spamming can prevent a site from being indexed.
Hosts: For submitting the sites, the web host needs to be very reliable. If a host remains down for most of the time then it would affect indexing of your sites. A site might be even removed from the database of the search engine. It is very important for your host to respond timely when the spiders crawl the pages.
