ChatGPT rival Anthropic’s Claude can now read a novel in under a minute

A new element of engagement. 

ChatGPT rival Anthropic’s Claude can now read a novel in under a minute.   By Kyt DOTSON

OpenAI LP rival chatbot maker Anthropic today said that it’s upgrading its own generative artificial intelligence with a much longer memory to take on the internet’s darlings ChatGPT and Google LLC’s Bard.

Since its debut late last year, generative AI has caught the attention of users and businesses for its ability to hold human-like conversations and pull insightful information from documents. 

Although these AIs are trained on vast amounts of data, one issue is that conversations with them are limited by the amount of memory that they can handle at one time, this is known as a “context window,” or how much text an AI can handle before it begins to “forget” what was being discussed earlier. This is the same reason that certain AI chatbots, such as Microsoft Corp.’s Bing Chat, have a limited number of responses built into them.

These AIs generally don’t operate in terms of words, and instead use tokens, which break down the text sent to them into parts of words and phrases, and Claude has been upgraded from 9,000 tokens to 100,000 tokens, which corresponds to approximately 75,000 words.

“This means businesses can now submit hundreds of pages of materials for Claude to digest and analyze, and conversations with Claude can go on for hours or even days,” Anthropic’s team said in the announcement.

In comparison, ChatGPT’s context window is approximately 4,000 tokens (or about 3,000 words) and OpenAI’s GPT-4 has a context window of 32,768 tokens (around 25,000 words).

To test the greatly enhanced memory for Claude, the Anthropic team fed the entire text of “The Great Gatsby” into Calude-Instant, a low latency version of the chatbot – but the developers modified one line to see if the AI would catch it. The AI read the entire novel and spotted the difference in under 22 seconds.

The novel is becoming a favorite testing ground for engineers building AI models, MosaicML Inc., a generative AI infrastructure and AI model provider, also used it to show off the capabilities of its MosaicML foundational model.

“Beyond just reading long texts, Claude can help retrieve information from documents that help your business run,” the Anthropic team wrote. “You can drop multiple documents or even a book into the prompt and then ask Claude questions that require synthesis of knowledge across many parts of the text.”

It is possible to work with smaller context windows and have an AI digest and provide insights for large documents. It means breaking them into smaller portions and having the AI summarize them down so that the summaries themselves can be stitched together into the smaller context window. The problem with doing this is that summarization necessarily leads to picking and choosing what to discard in a specific section of text and its context is then lost for comparison to later sections. That’s not necessary if the entire text can be held in memory at once.

With extremely large context windows this means that enormous documents can be processed all at once without losing sight of any data. It greatly simplifies digestion, summarization and explanation of dense material such as financial statements, research papers, legal filings, annual reports and more. Developers themselves could benefit from this by using it to read hundreds of pages of documentation and have the chatbot answer questions about technical documentation.  ... /