Web Searching with Search Engines

Ashley_Buckwell

What is a search engine? 

A search engine is an online tool that is designed to search for websites on the Internet based on a user’s search query. It looks for results in its own database, sorts them, and makes an ordered list of the most relevant results using unique and proprietary search algorithms. Although hundreds of different search engines have been developed, the general principles of searching and providing answers are the same across all of them. 

How do search engines work?

Although search engines have become more complex over the years, they still follow an original formula: crawling and indexing all the data on the web so that they can present a set of results for any given search query that are ranked by relevance. They do this in three steps: 

1. Crawl. Web crawlers, also known as spiders, are bots that are constantly crawling the Internet, finding new sites, and identifying new links. Crawlers also send text from every website to an index to be analyzed. 
2.  Index. The data that crawlers gather is analyzed, organized, and stored in an index so that the engine can find information quickly. Like the index found in the back of a book, a search engine index includes an entry for every word on every indexed web page. 
3.  Rank. A search engine uses proprietary algorithms to present the user with a list of results prioritized by what it thinks is the best answer to the user’s search query. Ranking algorithms are kept highly confidential as they are what differentiate one search engine from another. 

How do search engines rank results? 

Search engines want to present a list of results that the user will click on, and they use a variety of factors to rank results according to what they think are most engaging: 

• Keywords. Search results should match at least some of the words in the query. 
• Page content. Search engines prioritize high-quality content by analyzing the length, depth, and breadth of web pages. 
• Backlinks. Backlinks, or mentions of one website on another website, can be seen as a vote towards the authority of that site. Backlink ranking rates pages based on how many other sites link back to that site, and how highly those sites rank. 
• User information. Personal information, such as the user’s search history and location, are used to serve results that are uniquely relevant. 

Search engine optimization

Besides providing useful information for their users, search engines can also help businesses promote themselves and increase their relevance and ranking in search results. An important part of the marketing strategy of any business includes optimizing online presence and ensuring relevant search queries drive traffic to their webpages. The sum of all the practices and techniques a business will do to improve their search rankings is called search engine optimization (SEO). 

At a basic level, SEO activities can be divided into two categories:

• On-page SEO. This refers to building content that will improve the business’ search result rankings, by incorporating keywords, using accurate metatags, and crafting appropriate titles and headings. 
• Off-page SEO. Is the optimization done off the business’ website, such as earning backlinks and building relationships so that others want to share their content organically.

Why are search engines important?

Search engines essentially act as filters for the wealth of information available on the Internet. They allow users to quickly and easily find information that is of genuine interest or value. Search engines are so important and ubiquitous in contemporary culture that the verb “Google” has been an entry in most dictionaries for many years. 

However, this penetration into our lives has had unintended consequences. Search engines control what users see and filter results through proprietary algorithms. The details of how sites are ranked are not available to the public, and the algorithms can be programmed to prioritize specific topics or sources according to the search engine’s interests rather than the user’s.

Furthermore, all users of search engines are subject to increasing levels of surveillance and monitoring. Every search query is recorded, linked to the user’s real identity, and stored in perpetuity, along with other metadata such as the IP address and the date and time of the search. It remains to be seen how such information can be used in the future, for better or for worse. 

Types of search engines

Mainstream search engines like Google are usually top of mind, but other types of search engines are growing in popularity: 

1. Mainstream search engines: Google, Bing, and Yahoo! are all free to use and are supported by online advertising. 
2. Private search engines: are becoming ever more popular as people’s awareness of privacy concerns grows, and include anonymous, ad-supported search engines and ad-free, paid search engines, like DuckDuckGo, Brave, Neeva, and Kagi. 
3. Vertical search engines: narrow a search to one topic category or one dataset, rather than search the entirety of the web. Examples include: 
   1. The search bar on sites like Amazon and YouTube. 
   2. Google Scholar, which indexes scholarly literature across publications. 
   3. Searchable social media sites and apps like Pinterest.
4. Computational search engines: WolframAlpha answers math and science-related queries. 
5. Metasearch engines: also known as search aggregators, these index and collate results from mainstream search engines. Dogpile and Searx are popular metasearch engines. 

7 Notable search engines

Search technology has improved since the development of the first search engine. Here are the major players today: 

1. Google. The most popular search engine by far, with a clean look that is quick to load and a backlink-based ranking algorithm that made it famous starting in the 1990s. 
2. Bing. The second most popular worldwide, Bing features scenic photography on its homepage. 
3. Startpage. Acts as a proxy that allows users to obtain Google search results without providing trackable personal information and metadata. 
4. DuckDuckGo. A private, ad-supported search engine. 
5. Searx. A free, open-source, hackable, privacy-respecting metasearch engine. 
6. Kagi. A premium search engine with a paid subscription model that is 100% free of ads and tracking.

Since the launch of the first search engines in the 1990s, the field’s leaders have innovated on search technology to serve more and more needs with a single interface, and will continue to do so for the foreseeable future.

*The opinions reflected in this article are the sole opinions of the author and do not reflect any official positions or claims by Acer Inc.

About Ashley Buckwell: Ashley is a technology writer who is interested in computers and software development. He is also a fintech researcher and is fascinated with emerging trends in DeFi, blockchain, and bitcoin. He has been writing, editing, and creating content for the ESL industry in Asia for eight years, with a special focus on interactive, digital learning.