Nvidia Unveils LLM-based Engine for Safer AI Chatbots
- By Paul Mah
- May 02, 2023
Nvidia has released NeMo Guardrails, an open-source software designed to provide safer AI chatbots and other generative AI applications.
Large language models (LLMs) have captured headlines since the release of ChatGPT in November last year. However, LLMs are prone to hallucination and can be tricked into generating output deemed unsafe or offensive.
The difficulty in harnessing LLM applications safely and securely has resulted in enterprises being hesitant to adopt them for tasks such as customer support and other customer-facing services.
NeMo Guardrails
To address this, developers can leverage NeMo Guardrails to establish three types of boundaries for AI-powered applications, based on topical, safety, and security guardrails.
In a blog announcing its release, Nvidia states that the product is the result of several years of research.
Topical guardrails prevent apps from discussing unrelated subjects, while safety guardrails ensure the AI responds with accurate and appropriate information from credible sources. Security guardrails restrict connections to only trusted external third-party applications.
How does NeMo Guardrails work? According to Nvidia, a Python-based contextual dialog engine tracks the state of the conversation to prevent LLMs from executing malicious code or making calls to an outside application.
“The concept of security is becoming more and more important as large-language models are allowed to connect to third-party APIs and applications. This can become a very attractive surface for cybersecurity threats,” said Jonathan Cohen, vice president of applied research at Nvidia, during a virtual press briefing
“Whenever you allow a language model to execute some action in the world, you want to monitor what requests are being sent to that language model and what that language model is doing in response and provide a place to implement all of these sorts of checks that would indicate different kinds of attack and security threats.”
NeMo Guardrails itself relies on a specialized LLM for checks. Cohen says a model designed just for fact-checking can be less expensive than a general LLM.
NeMo Guardrails is available on GitHub as open-source code and can be easily integrated into existing tools and platforms, such as the toolkit LangChain and the automation platform Zapier.
Enterprises can also obtain it as a complete and supported package via the Nvidia AI Enterprise software platform or access it through Nvidia AI Foundations.
Image credit: iStockphoto/Tharakorn
Paul Mah
Paul Mah is the editor of DSAITrends, where he report on the latest developments in data science and AI. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose.