Generative AI models can produce tremendous results for certain applications–but they are also notorious for confidently, sometimes convincingly, making mistakes (or “hallucinations”) like suggesting people eat rocks five times a day or add glue to pizza. A new open source AI model called Lynx, developed by nascent AI evaluation company Patronus AI, aims to fix this problem. The model promises a faster, cheaper and more reliable way to detect such hallucinations without human help.
Cofounders Anand Kannappan and Rebecca Qian, both ex-Meta AI researchers, claim that the new model is more accurate than other leading AI systems, like OpenAI’s GPT models and Anthropic’s Claude 3 model, in detecting factual inaccuracies. To accomplish this, the company fine tuned Meta’s most advanced large language model, Llama 3, by showing it 2400 examples of hallucinations and their corresponding correct responses.
Before starting the company in September 2023, the duo spoke with about 60 company executives and found that their worst fear was launching an AI product and making headlines for the wrong reasons. CEO Kannappan hopes that Lynx can help assuage those concerns. He thinks of it as a “coach” for other AI models that can guide them to be more accurate. The goal is that its customers that are rolling out AI applications could use the Lynx to uncover hallucinations during development rather than fixing blunders after they have already launched.
“One of the reasons why Rebecca and I started the company was this concept called scalable oversight,” Kannappan said. “And it was about how humans can continue to supervise systems that far outperform them. And the only way you can do that is if you have a really, really powerful AI that evaluates AI.”
That’s a contrast to the way AI products are stress tested now before shipment, he continued, which involves a range of techniques. One of those is “red teaming,” which involves manually hacking AI models to expose vulnerabilities that might lead to mistakes. Other development teams use AI models like GPT-4 to catch hallucinations, Kannappan said, who criticized this approach as “literally GPT-4 evaluating GPT-4 itself.” That’s a problem, he explained, because general purpose models like GPT-4 weren’t specifically designed to catch errors. Lynx, on the other hand, was taught how to reason why an answer is wrong as it was fed more context, said Qian.
“We provided examples of incorrect answers and gave the specific financial calculation or medical citation that showed why the response is wrong,” she said. This is a more effective approach because the model is provided with additional background information to better catch similar mistakes.
The company has also released a new benchmark called HaluBench, which rates how well different AI models can catch hallucinations in model output, especially across legal, financial and medical domains. That benchmark shows that even Lynx isn’t perfect–it scores about 88% accuracy–but it does outperform most others, Kannappan said.
In March, Patronus AI also launched Copyright Catcher, a tool that detects when popular AI models like OpenAI’s GPT-4, Anthropic’s Claude 2 and Mistral AI’s Mixtral produce copyrighted content. The tool caught such models regurgitating entire paragraphs from books like Michelle Obama’s Becoming and John Green’s The Fault in Our Stars.
The company has also developed other tools that evaluate model performance across particular domains. For example, there’s FinanceBench, which is used to evaluate how well different LLMs answer financial queries; Enterprise PII, which helps companies use to detect whether AI models are exposing their sensitive and confidential information; and Simple Safety, which evaluates LLMs for safety risks such producing harmful responses related to suicide, child abuse and fraud.
All the work is geared towards the mission of the company—making sure LLMs aren’t producing bad results that people end up relying on. “When the model hallucinates, it still produces output that sounds plausible,” Qian said. “That ends up leading to misinformation.”
MORE FROM FORBES
