How to Accelerate Customer Service Automation with Generative AI

What is Generative AI?

Generative AI is an abbreviation of Generative Artificial Intelligence and is also referred to as GenAI.

Artificial intelligence refers to any technology that enables machines (and more specifically, computers) to do things on their own – e.g., recognize faces, analyze data, and understand text.

Generative AI is a specific type of artificial intelligence that enables machines to create original content. That content can be:

• Text
• Images
• Audio
• Video
• ...and more

Generative AI was popularized by Open AI’s introduction of ChatGPT in November, 2022. Read a full history and description of generative AI here.

How does Generative AI work?

To be concrete, we’ll focus on generative AI in text, and then generalize to all generative AI.

Generative AI requires:

• Training Data: Text-based generative AI requires a massive amount of text to train on. Open AI and other generative AI companies have basically crawled the entire public internet of text to train on.
• A model: Text-based generative AIs use all the training data to create Large Language Models (“LLMs”). LLMs are large algorithms that fundamentally enable 2 things: (1) the ability for the machine to comprehend the meaning of any text input, and (2) the ability to predict the next word (and the next word) in a conversation. This allows LLMs to generate human-like sentences and paragraphs with proper grammar and tone. The 2 capabilities combined allow LLMs to create coherent responses to any input or prompt.
• A lot of tuning, feedback and supervision: LLMs aren’t only algorithms. They require a significant amount (thousands of hours) of testing and feedback, i.e., people entering prompts into the LLM, reading the output, and providing feedback on the quality of the output.

The result of all this work is an LLM that can respond to any text-based prompt or question and seemingly converse with a human.

Voice-based generative AI uses a voice-to-text and text-to-voice technology, in addition to an LLM. The voice-to-text converts a voice prompt to text and sends the prompt to the LLM. The LLM then generates a response in text. The text-to-voice converts the text response back into a voice.

Image and video generative AI models are similar to LLMs. They train on a massive amount of image and video data; they depend on an LLM to comprehend a text prompt; and the model creates an original image or video based on the prompt.

How is Generative AI different from Conversational AI?

“Conversational AI” is AI technology that existed before 2022, prior to the popularization of generative AI.

Conversational AI maps the user's input to a specific intent or topic. It does this through either:

• Simple keyword detection ("if someone's question includes the phrase "reset password," the user's intent must be "I want to reset my password.") or
• More advanced Natural Language Processing (NLP). Think of NLP as more fuzzy logic matching -- e.g., if someone enters, "I need a new password," NLP will determine the user's intent to be "I want to reset my password."

The real challenge with conversational AI is, you need to create and maintain (1) an inventory of every intent your customers might have and (2) a pre-created canned response for every intent.

Relative to generative AI, there are 3 significant drawbacks to conversational AI:

(1) It may or may not determine the right intent (and what if a question has multiple intents?) Generative AI uses LLMs to determine the intent(s) of the customer's question/statement, and LLMs are by far and away the most advanced and best technology to truly understand what any customer is saying (and in 100+ languages).

(2) The canned response will look and feel like a canned response. Generative AI uses the LLM to compose a highly customized response to the customer's question. Just like ChatGPT, a customer service AI agent that uses generative AI will interact with a customer just like a human agent will.

And most importantly,

(3) Generative AI requires much less time for implementation and maintenance. There's no list of intents you need to manage. There's no list of canned responses you need to manage. You simply need to point the generative AI to your knowledge (i.e., support KB, FAQ, resolved tickets, etc.)

Read this article for more information on generative AI versus conversational AI.

Is Generative AI the same thing as ChatGPT?

ChatGPT uses generative AI and is the most popular generative AI application. However, there are many other generative AI applications beyond ChatGPT.

Is ChatGPT an LLM? If not, what is it?

No, ChatGPT is not an LLM. ChatGPT is a chatbot that interfaces with Open AI’s LLMs (GPT-3.5, GPT-4, GPT-4o).

The free version of ChatGPT interacts with GPT-3.5.

ChatGPT Plus interacts with GPT-4 and GPT-4o, which are more advanced and have more human-like responses.

In Generative AI, how does a chatbot interact with an LLM?

The basic functions of a chatbot in Generative AI are to:

• Provide a user interface for the end-user to enter a query or response
• Maintain and provide context – i.e., “remember” everything the user (and LLM) has said within a given conversation and feed the full conversational context back to the LLM, each time a customer adds to the conversation. (A chatbot might also maintain a full history of conversations.)
• Display the LLM’s response to the user.

What are Generative AI hallucinations, and what causes them?

Hallucinations are a built-in feature of generative AI, not a bug.

LLMs (in text, or their equivalent in images and video) are mathematical models that simply predict the next most probable word in a conversation.

Note, however, that the next most probable word doesn’t guarantee that the full sentence or paragraph generated by the LLM is factually correct. The same is true for images and videos created by generative AI as well.

For some industries or applications, like fiction writing, hallucination is a good thing. Hallucination allows a generative AI to be creative.

For applications like pre-sales shopping assistance and post-sales customer service, hallucination is a critical issue. You don’t want your Sales Assistant AI to invent products, features, or prices, and you don’t want your customer service AI to provide incorrect (and outright misleading) information to paying customers.

Read more about AI hallucinations in this article, or watch this brief video:

Is there any way to prevent Generative AI from hallucinating?

No and Yes.

Hallucination is a built-in LLM feature. So there’s no way to prevent an LLM from hallucinating.

However, good chatbots/generative AI solutions that interface with LLMs can minimize or even eliminate hallucination by:

• Passing the right context into the LLM – the context can include content from your knowledge base that is most relevant to the conversation
• Proactively detecting hallucinations by comparing the LLM’s response to the most relevant content in your knowledge base

The process above is called Retrieval-Based Augmentation (or “RAG” for short). Read this article to learn more about RAG, or watch this brief video:

Can I just let my contact center agents use ChatGPT?

No, your contact center agents shouldn’t just use ChatGPT.

ChatGPT can in theory be useful as:

1. A brainstorming tool, e.g., “Come up with different ways to say the following…”
2. A reformatting tool, e.g., “I’ve written the following email to a customer. Please edit it and make it nicer and more professional.”

However, ChatGPT has the following serious limitations:

1. OpenAI can train on anything entered into ChatGPT. As a result, agents using ChatGPT can result in proprietary information “leaking” outside the company.
2. ChatGPT was trained on the entire internet, not on your company-specific knowledge. So ChatGPT can create responses that are incorrect for your business.

Can I let my agents use ChatGPT Enterprise?

Open AI offers ChatGPT Enterprise, which is a “private” version of ChatGPT.

1. Employees can use ChatGPT Enterprise without leaking proprietary information.
2. However, ChatGPT Enterprise still is trained on the entire internet and will generate responses that aren’t specific to your company.

Is Generative AI secure?

It depends. Not all software solutions are created equal.

If you use any third party generative AI solution, you should make sure that all aspects of the solution are SOC 2 Type II compliant.

SOC 2 Type II compliance is a third party audit that ensures your vendor follows SaaS industry best practices to ensure the privacy, security, confidentiality of your data, as well as the availability and processing integrity of its operations.

Will all information sent to OpenAI become public knowledge?

It depends.

OpenAI owns all information entered into ChatGPT, and OpenAI can train on that information. This can result in proprietary information “leaking” to the public via ChatGPT.

Any proprietary information entered into ChatGPT Enterprise and directly OpenAI’s APIs (GPT-3.5, GPT-4, GPT-4o, etc.) will remain proprietary, and OpenAI will not use the information for training purposes.

Does Generative AI only answer questions?

Generative AI can do much more than just answer questions.

Generative AI can write code, create images and videos, analyze data, and reformat existing content.

In addition, generative AI applications can also interface with existing APIs and enable customers or employee self-service. Users can interface with those APIs in a conversational manner, both to read from existing databases (like products, prices, and inventory) and to write to databases (create, edit, or cancel an order).

Does Generative AI integrate with existing contact center solutions?

Many enterprises have already invested in a help desk solution (like Zendesk, Freshdesk, or Salesforce Service Cloud), a contact center as a service (CCaaS) solution like NICE, Five9, Genesys, or Sprinklr, or unified communications as a service (UCaaS) solution like Nextiva, RingCentral, or Vonage.

These enterprises might naturally assume that they need to use a generative AI solution from an existing contact center vendor.

A generative AI solution should be an independent decision from your contact center solution. You should treat a generative AI solution like you might treat a new contact center agent. You need to train and supervise that agent, and you need to give that agent a “seat” in your existing contact center solution.

A good generative AI solution should work seamlessly with your current contact center solutions, just like a new contact center agent would. It should complement, not replace, your current solutions—allowing your team to leverage its existing systems while maximizing efficiency with AI-driven automation.

Can Generative AI handle sensitive customer information?

Not all generative AI applications are created equal.

Your agents should not be entering sensitive customer information into ChatGPT.

Your third party generative AI solutions (and any vendors they use as subprocessors, like an LLM) should, at minimum, be SOC 2 Type II compliant. SOC 2 Type II compliance doesn’t automatically prevent all data breaches/data leaks, but it is a minimum level of protection you should expect in a generative AI SaaS solution.

If you or your customers are based in Europe, you should ensure that your generative AI solutions are GDPR compliant, and if your generative AI solution is storing data in the US, that they are participating in the Data Privacy Framework, which is enforced by the Federal Trade Commission.

For an additional layer of privacy/security, you can potentially ask your generative AI solution provider to (1) be on the same public cloud that your infrastructure is hosted on and (2) have a virtual private connection between your infrastructure and their infrastructure.

For even greater privacy, you could host and fine-tune an open source LLM and eliminate the need to send sensitive customer data out to a third party. This will require significant machine learning expertise and IT operations expertise.

Continue reading Chapter 2 - Generative AI Applications in Customer Service Automation