Sunday, February 19, 2012

Searching for Reality is Becoming More Difficult with Google and Bing

An interesting change is taking place in the world of search. Google - the undisputed king of the hill in 2012 - is about to change everything for the worse. Perhaps it's a natural evolutionary step for search engines, but reality is getting harder and harder to find on the Internet.

Google has never been good for academic research, but it's getting worse and bringing Bing along for the ride.

Let me give some background.

First, some stats. Statistics vary, but I'll just pick one source and you can go side-check it with your own research. As of February 2012, eBizMBA (http://www.ebizmba.com/articles/search-engines puts the battle like this:

- #1 Google with estimated 900,000,000 unique monthly visitors
- #2 Bing with estimated 165,000,000 unique monthly visitors
- #3 Yahoo! with estimated 160,000,000 unique monthly visitors
- #4 Ask with estimated 125,000,000 unique monthly visitors

- Everyone else 33,000,000 or fewer, dropping quickly to less than a million for #15 Alta vista.

Lesson #1: It pays to be number one. Google has WAY more visitors than ALL of their competition combined.


Second, Some history. In the early days of the Internet, there was no world wide web. That is to say, there was no graphical user interface to the Internet. We used tools like telnet and later FTP. A brilliant young man named Scott Yanoff compiled a list called Scott Yanoff's list. It was his attempt to list every resource on the Internet in a text file. That actually made sense when the Internet was small.

Most people received the list via email or Telnet from within their university. If you had access to a service called Finger, you could "finger [email protected]" and capture the list. By the time the Web came along and started to grow in the early 1990's, actual search tools had come into existence. Archie and WAIS allowed academics and other resourceful people to scour machines and find stuff.

True search engines came along with Excite, Alta Vista, and Yahoo. These search engines use "bots" to go find information and index it. That led to amazing attempts (mostly by the porn industry) to trick the search engines into higher rankings. That in turn led to search engines becoming more and more sophisticated about finding what people were really looking for. Later, everyone who claimed to be an SEO Wizard made the battle even more difficult.


Sidecheck: As an academic, trying to find the "reality" of what's going on in the world, search engines have never been very good. In these early days, if you could filter out the porn hits, you had a pretty good idea. But it was more likely than not that the information you wanted was not on the Internet! Now we assume almost everything is on the Internet (it's not). But back then you had to assume that most information was not.


Third, Enter Google. Google changed the game. Instead of trying to simply match "keywords" with search terms, Google did deep-dive analysis of what web sites were trying to say. For example, if I have a link that reads Best Sandwich in The World and I link to your sandwich shop, Google knows that I think your sandwiches are the best. That increases your ranking for the search term "Best Sandwich."

If I just link to your site and Cousin Larry's Sandwich Shop, that does you very little good. It increases your in-bound links. Good. It lists you by name. Good. But if people are not searching for you and only searching for the best sandwich, it does you no good.

Important lesson. When you ask people to post a link to your site, you MUST give them a recommended link format that maximizes your keywords. For example, I might ask you to link the term

"Managed Service Agreements"

to

www.smbbooks.com/products/smbpal03.htm

Google interprets that and knows that someone considers that page to contain authoritative information on Managed Service Agreements. This is HUGE in the search business. It means that Google was actually using the evolving sense of subject matter authority to return search results.

Of course the SEO spammers continue to trick the system. Google keeps evolving. And the uniqueness that catapulted Google to the top has been eroded.


Sidecheck: In the academic search for "reality" Google plays an interesting role. If you're looking for who is the best, who knows the most, and who is considered the one true source of information, Google might get you there. But if you're looking for hard, cold facts, Google is a step in the wrong direction. It's more like a popularity contest than a tool for finding the truth behind the reflections of reality we see every day.


Fourth, Enter Bing. Bing took a different approach. Bing was introduced as a "decision engine." In other words, they assume that you're looking for something so you can make a decision. Where to live, which train to take, which shoes to buy, etc. So Bing returns a variety of results that are supposed to help you make a decision.

Clearly, this is not helpful in the search for who really invented the telegraph or how many people have died from Dihydrogen Monoxide abuse.

Luckily, Bing has evolved to be more or less just another search engine. So it returns much more useful results than it did three years ago. In the search for "reality," however, it has had no effect.


Fifth, Google is changing again. But finding the real, live "truth" behind a topic will now be impossible. Google's latest move to change their "privacy policy" is really a move to improve their search engine and money-making efforts. I'm not opposed to that in any way. But you should be aware of the effect it will have on your searches.

Google will now try, whenever possible, to gather up your activities from various Google products and throw them into consideration when you search. So if you searched for certain things in the past, that will affect your searches in the future. If you read certain blogs, post certain pictures, or post replies or comments on web sites, that's taken into consideration. The keywords you use in your Gmail, the Google+'s you plus, and the Google Docs you use will all affect your search results. All of your activity will be taken into consideration in order to tailor the advertisements you see and the search results you receive.

Google's goal is to give you a world wide web experience that includes the things you like, the things you enjoy, and the things you're familiar with. That allows them to give you unique and interesting results that no one else will experience. If the world is defined as spending money in order to express your interest, then Google wants to give you a very fulfilling experience.

The web will literally be molded to fit your needs and desires.

And that's cool, but it seems to me there there are two big reasons this is bad. First, it means that we will see fewer and fewer sites that are "interesting" because they are unexpected. Our experience will become more and more homogenized as we use Google products more and more. For many people, the fun of the Internet includes stumbling onto things that are new and fun and different. It sounds like we'll see less of that.

The second reason this change is bad is that reality will be even harder to find. Because our future search experience will be so completely colored by our past Internet experiences, Google will move further away from being a useful research tool. Now, granted, Google doesn't want to be a research tool. It's not their mission. They've never presented themselves as that.

When my daughter was little, I told her time and time again that Internet research needs to be backed up with real, hard core, facts. Anyone can put up any crap they want on the Internet. And if someone believes it, then it becomes just another competing version of reality.

Most of us are not academics. But we all do research from time to time. Every good anthropologist and social scientist knows that we change reality simply through the process of observing it. So maybe computer-aided search tools will never get us to the "truth" in the real world. Maybe reality will always go to the highest bidder or the most popular page.

But when we really need to do hard-core research, we need to choose the right tools for the job. Popular, commercial search engines are NOT the right tools for this job. When you're serious about serious background information and fact-seeking, there are many alternatives. Luckily, you have hundreds of search engines to choose from. Some of the best are highly focused on one specific topic or set of resources.

A great list of search engines can be found here: Noodle Tools: Choose the Best Search for Your Information Need. It is also a good idea to talk to a Librarian. They do this for a living.

Please don't assume that Google, Bing, or Yahoo will give you the information you need. For many of us in marketing, they are just right. But for real research, they are as far from reality as you can get.

:-)

2 comments:

  1. I was visiting the author profile for Karl W. Palachuk on Amazon.com this morning when I saw the title for this blog post and eagerly came to see what you had to say on the matter. I had a feeling it would be quite different from what the SEO gurus teach, and I'm looking for a fresh perspective. (For example, I'm tired of the way they use scare tactics about something called Panda to get people to buy their SEO fixes.)

    I enjoyed learning about the origins of the Internet and how it sprung from the need for hard-core academic research. (Ironic, considering what it has become and how most of its users these days are oblivious to its early history.) I'm far from using it for hard-core research myself—I enjoy the way it enhances my lifestyle in so many ways and I also use it to run my creativity-based business—but I don't look forward to having that homogenized even more than it already is.

    I used to resist being a part of the Google cluster. But eventually I was absorbed by it and now I'm a Google girl (Gmail, Google Docs, Feedburner, yada yada), even though I see their monopoly as putting too much power in the hands of a company that can negatively affect people's livelihoods with their policies.

    This was a well-written and refreshing article. I'm bookmarking it and will revisit later to absorb it even more deeply.

    ~ Milli Thornton

    P.S. The Noodle Tools link looks very useful. I may never have run across that in my Google-directed searches ;~) and it sounds like it'll broaden my search scope with more options, while also narrowing it for specialized searches. Thank you!

    ReplyDelete
  2. Well, thanks for dropping in!

    I appreciate your comments. We're all getting absorbed into the Google Cluster (and the Microsoft Cluster). Sometimes it's just good to be aware of that fact.
    -kp

    ReplyDelete

Feedback Welcome

Please note, however, that spam will be deleted, as will abusive posts.

Disagreements welcome!