As I have already said, it is a common delusion that Google=Internet. No, it is not and it will never be. There are probably hundreds of reasons why this is so but I hope that even some of them are convincing enough:
- Search engines and their indexing algorithms might be very powerful but it will never be possible to include into their database every single page that exists on the Web. At least because new pages appear every instant, while search engine crawlers do not visit sites so often – sometimes a site is revisited one or two months after the previous visit and all the pages that appeared after the last visit will not be indexed. You see why search engines do not deliver real-time weather, stock, or news information?
- The site, on which the page you want resides, requires registration and/or a fee and there is nothing on Earth that the search engine can do to index such a site. If the site requires registration, after you register, you will be able to find the stuff you need but there is no other way to know if the stuff is there besides visiting the site and registering. It is a similar situation, when the site of interest to you is locked inside a database and search engines cannot access it because of that but when you go there, the site is searchable by humans and you can get what you want.
- Generally, search engines prefer static to dynamic sites and are reluctant to index dynamic ones (these are database sites where pages are generated dynamically on a user’s request). While dynamic sites are more powerful from a technical point of view, they are not the favorites of search engines and often are either not indexed at all, or only part of their content is crawled. If the URLs of the pages have question marks and other special symbols (like non-ASCII characters), then it is a good bet that this page will not be indexed by search engines, or at least the major ones.
- Due to a violation of the fair play rules, search engines have excluded (temporarily or permanently) particular sites from their listings. This situation is much worse for site owners than for ordinary users but it is another reason for you – the ordinary user – not to be able to find the stuff you want. It is a little consolation that after some time the site will be indexed again. Even if this happens, you will spend some time in the shadows of the Invisible Web.