Understanding Search Engines
- Search Engine Market ShareSearch Engine Market Share - StatCounter, one of the best data sources for this
- Answer the Public - A fun tool to see the questions people are asking search engines
- What is a web crawler? - Great Wikipedia article on crawlers/bots
- List of Google Products - All the ways Google is capturing your data :)
- TensorFlow - Just one piece of Google's machine learning puzzle
- How Google Makes Money
So, now that you know what SEO is, and how it came to be, the next step is understanding what a search engine is…and isn’t.
When speaking of search engines, globally, this is basically it. There aren’t any other players in the space right now that matter, and out of these 5, Google is BY FAR the most dominant. How dominant?
This site shows the rough distribution of global search market share worldwide. At 91.64% of global search engine market share, nobody else can hold a candle to Google’s flame.
Search engines are, for all intents and purposes, modern day oracles. You ask them questions, and they provide you with answers. They do their best to give you the correct answer, regardless of who you are, where you are searching, and how you phrase your query. And they do it in a fraction of a second. Millions of times per day. In hundreds of languages. For people all over the planet. This is mindbogglingly complex.
First and foremost, search engines use Bots or Crawlers. A bot is just a piece of code that follows links and captures data. That data is downloaded, stored, and then analyzed. Google has crawled TRILLIONS of web pages, over the course of 20 years. That is an enormous amount of data…and that is just one source.
Google is capturing data from SOOOOOO many places. And they augment this data with other data from aggregators and data brokers as well. I think it would be safe to say that no company on the planet has more data and insight into all facets of humanity than Google does. I doubt there’s a single person who accesses the internet on the planet who doesn’t use at least one Google product, and most likely use many.
The odds are good you’ve heard the media chattering about Artificial Intelligence or Machine Learning over the last couple of years. It’s going through something of a Renessaince. Google takes all of this data and uses machine learning to parse it and connect the dots, so that they can accurately answer whatever query you throw their way. This bit, the machine learning, is their secret sauce. We’ll dive into that a bit more shortly. But for now…why does Google do this? Why do they go to such great lengths to perfect search?
To sell these. Ads. Search ads alone make up almost 80% of their revenue, with most of the rest coming from display ads and the Android app store. So, search engines exist to sell ads. And they sell ads by getting eyeballs on their site, doing searches. And they do that by providing accurate, useful results, regardless of what you search for.
And this, at the end of the day, is the key to SEO. Understanding not only how a search engine works, but why they do what they do. When you understand the motives of a thing, it makes it far easier to anticipate how that thing will act, and react.