Jakub Zavrel of Zeta Alpha talks putting GPT on a leash at KMWorld 2023
What are the best ways to apply generative models inside the enterprise? Generative language models like GPT show an astonishing ability to summarize, digest, and explain information faster than any of us can, but are also known to lack the knowledge to answer questions that matter to the work inside your company and can be prone to “hallucinate” when lacking knowledge.
At KMWorld 2023, Jakub Zavrel, CEO, Zeta Alpha discussed the core concepts of retrieval in augmented generative AI, including neural and vector search, prompting frameworks like LangChain, tips and tricks for prompt engineering, and proprietary and open source large language models.
High-quality search has been a bottleneck to enterprise knowledge management for years, but new developments in AI have led to a much better language understanding, especially for complex natural language discovery queries, allowing enterprise search functionality beyond keyword search, without the efforts in named entity recognition and linking, without domain specific ML models, and without expensive ontologies or knowledge graphs.
Zeta Alpha offers a Knowledge Assistant that can guide users during the onboarding process to new topics or projects. Questions will lead to answers that can be compared to talking with a knowledge expert, he explained.
“What’s really changed in the last year is this idea of reinforcement learning from human feedback in which these models are finetuned to follow instructions that align with human preferences,” Zavrel said.
The first thing that needs to be put “on a leash” are expectations regarding productivity, Zavrel said. Exposure to ChatGPT increases job satisfaction and confidence when surveying students. Another study on consultants also found an increase in productivity. However, he noted that humans have difficulty starting from scratch with an idea. By prompting these AI technologies, it’s easier to start working on something with an idea in mind.
There are increasing concerns while using GPT though, he explained. They provide very convincing outputs but with “hallucinations.”
“It will even invent research articles, web pages, and even wiki articles that support its point,” Zavrel said.
With retrieval augmentation generation (RAG), a system that works between search indexes and an LLM system, users will be able to fact check all information coming out of AI.
“This is the leash to prevent it from hallucinating,” Zavrel said.
KMWorld returned to the J.W. Marriott in Washington D.C. on November 6-9, with pre-conference workshops held on November 6.
KMWorld 2023 is a part of a unique program of five co-located conferences, which also includes Enterprise Search & Discovery, Enterprise AI World, Taxonomy Boot Camp, and Text Analytics Forum.