One of the things I appreciate and respect about Anthropic, the creators of Claude, is the transparency of their messaging and content. The content is easy to understand, and that’s a plus in this space. Whenever I visit their site, I have a clear picture of where they are and the plans for moving forward. OpenAI’s recent shenanigans have piqued my curiosity to revisit other chatbot tools. Over a month ago, I wrote a comparative discussion about a few AI tools. One of the tools I discussed was Claude 2.0. Now that Claude 2.1 has been released, I wanted to share a few highlights based on my research. Note most of these features are by invitation only (API Console)or fee-based (Pro Access only) and are not generally available now in the free tier. There is a robust documentation library for Claude to review.

The Basics

• Claude 2.1 is a chatbot tool developed by Anthropic. The company builds large language models (LLM) as a cornerstone of its development initiatives and its flagship chatbot, Claude.
• Claude 2.1 manages the API console in Anthropics’s latest release. This AI machine powers the claude.ai chat experience.
• In the previous version, Claude 2.0 could handle 100,000 tokens that translated to inputs of around 75,000 words.
• A token is a unit measurement of text AI models use to represent and process natural language. The unit can be code, text, or characters, depending on the method of tokenization used. The unit of text is assigned a numeric value fed into the model.
• Claude 2.1 delivers an industry-leading 200K token context window, translating to around 150,000 words, or about 500 pages.
• A significant reduction in rates of model hallucination and system prompts in version 2.1 means more consistent and accurate responses.

200k Tokens Oh My!

Why the increase in the number of tokens? Anthropic is listening to their growing community of users. Based on use cases, Claude was used for application development and analyzing complex plans and documents. Users wanted more tokens to review large data sets. Claude aims to produce more accurate outputs when working with larger data sets and longer documents.

With this increase in tokens, users can now upload technical documentation like entire codebases, technical documentation, or financial reports. By analyzing detailed content or data, Claude can summarize, conduct Q&A, forecast trends, spot variations across several revisions of the same content, and more.

Processing large datasets and leveraging the benefits of AI by pushing the limit up to 200,000 tokens is a complex feat and an industry first. Although AI cannot replace humans altogether, it can allow humans to use time more efficiently. Tasks typically requiring hours of human effort to complete may take Claude a few minutes. Latency should decrease substantially as this type of technology progresses.

Decrease in Hallucination Rates

Although I am interested in the hallucination aspects of AI, for most this is not ideal in business. Claude 2.1 has also made significant gains in credibility, with a decrease in false statements compared to the previous Claude 2.0 model. Companies can build high-performing AI applications that solve concrete business problems and deploy AI with the goal of greater trust and reliability.

Claude 2.1 has also made meaningful improvements in comprehension and summarization, particularly for long, complex documents that demand high accuracy, such as legal documents, financial reports, and technical specifications. Use cases have shown that Claude 2.1 demonstrated more than a 25% reduction in incorrect answers and a 2x or lower rate of mistakenly concluding a document supports a particular claim. Claude continues to focus on enhancing their outputs’ precision and dependability.

API Tool Use

I am excited to hear about the beta feature that allows Claude to integrate with users’ existing processes, products, and APIs. This expanded interoperability aims to make Claude more useful. Claude can now orchestrate across developer-defined functions or APIs, search over web sources, and retrieve information from private knowledge bases. Users can define a set of tools for Claude and specify a request. The model will then decide which device is required to achieve the task and execute an action on its behalf.

The Console

New consoles can often be overwhelming, but Claude made the commendable choice to simplify their developer Console experience for Claude API users while making it easier to test new prompts for faster learning. The new Workbench product will enable developers to iterate on prompts in a playground-style experience and access new model settings to optimize Claude’s behavior. The user can create multiple prompts and navigate between them for different projects, and revisions are saved as they go to retain historical context. Developers can also generate code snippets to use their prompts directly in one of our SDKs. Access to the console is by invitation only based on when this content was published.

Anthropic will empower developers by adding system prompts, allowing users to provide custom instructions to Claude to improve performance. System prompts set helpful context that enhances Claude’s ability to assume specified personalities and roles or structure responses in a more customizable, consistent way that aligns with user needs.

Claude 2.1 is available in their API and powers the chat interface at claude.ai for both the free and Pro tiers. This advantage is for those who want to test drive before committing to Pro. Usage of the 200K token context window is reserved for Claude Pro users, who can now upload larger files.

Overall, I am happy to see these improvements with Claude 2.1. I like having choices in this space and more opportunities to learn about LLM in AI as a technology person interested in large data sets. Claude is on my shortlist.

Originally published at https://mstechdiva.com on November 23, 2023.