This precisely what we wanted to do, to let our systems understand our meaning of words, rather than just what the meaning they have been assigned by language. But would that work? We wanted to assign the meaning that WE had assigned for that assemblage of letters. Assume this could be one by some preprocessing or postprocessing or meta assignment. Nevertheless this is still a big step forward to empower practical use of meaning. But where is AWS?
Semantics Beats Syntax By R. Colin Johnson who is a Kyoto Prize Fellow who has worked as a technology journalist for two decades. In ACM.
IBM, Google, and Microsoft are all poised to release semantic engines (algorithms using the meaning of words) to supplement their current syntax engines (using the spelling of words, such as the search engine BM25). Their common goal is to extend their natural language processing (NLP) capabilities into engines that rival human semantics (our understanding of what language, words/sentences, mean). In line with other contenders, including Amazon, Intel, and Oracle, these semantic engines offer machine-understanding of meaning aimed at enhancing searches, artificial intelligence (AI), human–computer interaction, question answering, and automatic narrative generation (from descriptions/explanations to prose/poetry).
Today's syntax-only engines are blind to the meaning of the keywords used to ascertain results. For example, a human understands that "where Alan Turing was born" means the same thing as "the birthplace of Alan Turing," and "the town where Alan Turing was delivered as a baby." Their syntax is different, having unique keywords "where, born"; "birthplace"; and "town, delivered," respectively. However, each phrase's meaning, or semantics, are identical ("London" is the answer to all three). People understand this immediately, but computers—not so much.
Consequently, all three companies are developing algorithms that understand the meaning of words. Google and Microsoft are both building semantic engines that add metadata to sentences (Google) or words (Microsoft) using clusters of processors running multiple deep neural networks (called transformers, which use massive parallelization).
IBM
For a decade, IBM has been extending NLP to artificial intelligences (AIs)—starting with its Watson (2011), the AI that beat human experts in the television game Jeopardy, and most recently with its Project Debater (2021). During this decade (2014), IBM also developed its Cognos transformer—a metadata-based algorithm for creating engines it calls PowerCubes, which can then be used with its Business Intelligence software. But for its semantic engine, IBM chose instead to augment neural networks with symbolic logic that cuts down the number of examples it requires to learn. Transformers, it discovered when developing Cognos, require larger example sets to learn, according to Forrester Research principal analyst Kjell Carlsson.
"IBM's semantics uses a much more efficient encoding of knowledge, enabling high performing enterprise use-cases to be built with significantly smaller training examples," said Carlsson. "IBM's semantics can also provide explainability of its conclusions by virtue of symbolic-logic reasoning, governance to block the model from using faulty logic, plus provides fairness that prevents models from learning discriminatory reasoning," said Carlsson.
Transformers, on the other hand, are all neural networks—end-to-end—without meaning instilled by symbolic logic, which also enables easier explanations for why a neural network comes to specific conclusions.
"IBM's neuro-symbolic approach," said Carlsson, "enables higher accuracy with less training data, plus it also enables engineers to 'teach' a model logical-relationships that domain experts know to be true, which is far more efficient than having these relationships be learned by transformers."
In the words of Salim Roukos, an IBM Fellow and the company's Global Leader for Language Research, as well as Chief Technology Officer of the company's Translation Technologies unit, "Large end-to-end neural models require significant amounts of data to perform well in a new domain. IBM is more focused on semantic parsing of human language to enable developers to build text-understanding applications. By leveraging the semantics of human language, very small amounts of data from the application domain are needed to enable understanding."
