AI Will Augment SEO

AI Will Augment SEO,    By R. Colin Johnson

Commissioned by CACM Staff, May 24, 2023

"I fully expect that future personalized AI-enhanced search engines will allow you to query any corpus of information by requesting that the results be limited to a specific basket of domains," said Kevin Lee, CEO of the eMarketing Association.

Search engine optimization (SEO) is an Internet marketing technique that analyzes how the algorithms in a search engine work, then add metadata (for instance keywords, links, and other embedded content) to boost the page's ranking on the search engine results page. According to business marketing company Clutch, there are over 15,000 search engine optimization (SEO) companies in the U.S. alone.

However, without using artificial intelligence (AI) to rank Web page content (the text and images), SEO companies are left to boost rankings on the basis of metadata alone, according to researchers at Germany's Hamburg University of Applied Sciences, who claim non-optimized but high-quality content may thus be outranked (appearing lower on the list of search results) by search-engine-optimized content of lower quality, but attached to more alluring meta-data.

"To improve search result ranking, SEO companies add metadata to Web pages that match queries to the ranking criteria of the search engine's results," said Sebastian Schultheiß, lead SEO researcher at the Hamburg University of Applied Sciences. "However, a Web page that complies with SEO criteria does not necessarily provide content of higher quality from the user's perspective. For example, if the content of a Web page is one-sided and therefore not objective, this lowers the information quality of the page from the user's point of view but has no effect on the search engine's ranking. As a result, Web pages with content of lower information quality can receive better rankings than a page of higher quality but lacking SEO."

Such regrettable outcomes were measured by Hamburg University researchers using real test subjects, as documented in a paper presented at last year's ACM Conference on Human Information Interaction and Retrieval (CHIIR). The content was confined to Web pages with medical data; the researchers found users chose to click on results at or near the top of the results stack, regardless of their medical efficacy.

"The potential danger is that Web pages with inappropriate content, but which make intensive use of SEO and are thus ranked higher, will be chosen over Web pages with higher-quality content and no SEO measures. Our results show that users consistently select those items prominently placed by SEO," said Schultheiß.

In a paper he presented at this year's CHIIR '23 conference, Schultheiß described how he is expanding this line of research to include not just the influence of SEO, but also the wider field of search engine marketing (SEM), which manipulates not just the metadata, but the content itself. Schultheiß also aims to research paid search marketing (PSM) services, where search engines permit marketers to pay cash to lift Web pages near the top of search results. Schultheiß also is expanding the scope of his research to the fields of health, politics, and the environment, to see if lower-quality content there is also attracting users' focus due to SEO, SEM, and PSM.

AI to the Rescue

The solution to raising the quality of search engine results, according to Google senior vice president in charge of search Prabhakar Raghavan, is artificial intelligence (AI). "With artificial intelligence," he said at the latest Google I/O meeting, "we are transforming search to be more helpful than ever before." For instance, later this year, Raghavan said Google will add a new way to scan store shelves that overlays metadata about the products on them in your camera screen.

SEO expert Kevin Lee, who is CEO of the eMarketing Association, agrees that AI is the solution to obtaining higher-quality results from search engines. According to Lee, having intelligent algorithms built into search engines will allow them to "read" Web page content, look up related information (such as user reviews), and improve the resulting search engine ranking on the basis of "qualitative" matches to a user's query, rather than depending solely on SEO, SEM, and PSM.

Lee says in the future, the best AI search engines will build domain-specific knowledge into specialized search engines. "Google, like every other search engine, whether it searches niches or the whole Internet, is already enhancing their algorithms with AI in an endeavor to provide more relevant results to every searcher," said Lee. "But academics and professionals have different needs, and so in many instances, generic search engines even with AI may not find the most relevant results for individuals."

Academics and professionals are on the cusp of search-engine specialization using AI that targets specific content—such as IBM's Watson Healthcare solution for medical doctors. Watson, IBM says, actually understands medical content well enough to match high-quality results to a doctor's desire for the best diagnoses for their patients. Google Health also is developing a physician's search assistant that uses AI to help make more accurate diagnoses. Its Medical Pathways Language Model (MedPaLM) is currently being tested "to augment a clinician's ability to detect breast cancer, to better understand skin conditions, and to more accurately sequence genomes," says Google team leader Alan Karthikesalingam.

For professionals in fields not served by specialized search engines, the future holds the ability to roll-your-own specialization using AI tools, according to Lee.

"I fully expect that future personalized AI-enhanced search engines will allow you to query any corpus of information by requesting that the results be limited to a specific basket of domains," said Lee. "AI will parse a natural language query with far more granularity than today's algorithms, and in the future perhaps even learn each user's unique preferences. Users who consistently select a particular type of result will be training their personalized search engine's AI to find those types of results in future queries."

Google this month is reportedly expanding its AI reach with personalization using a new tool called Magi, which it is adding to Google Lens (for searching with images) and Multi-search (for multi-media searches).

R. Colin Johnson is a Kyoto Prize Fellow who has worked as a technology journalist for two decades.