How to Accelerate Customer Service Automation with Generative AI

What are the costs associated with implementing a generative AI solution?

Generative AI costs will vary by vendor, but in general, the costs associated will be:

• Direct costs

  • Cost of integrations/custom work
  • Software licensing or usage-based costs

  Indirect costs

  • Ongoing cost of knowledge curation
  • Ongoing cost of AI supervision & feedback

Do I need to carve up all my knowledge into “snippets”?

It depends.

Some generative AI solutions may require you to organize your knowledge into smaller, tightly themed documents, or snippets. This can be time consuming and tedious.

Better generative AI solutions will not require you to organize your knowledge into snippets. Instead, the AI should automatically digest and organize the knowledge.

Will I need to translate all of my knowledge into different languages?

No, a best-in-class generative AI solution should not require you to translate your support knowledge.

A good GenAI solution will automatically detect the customer’s primary language, understand what the customer has asked, and be able to formulate a relevant and accurate answer in the customer’s language as well.

What support knowledge should I train a generative AI on first?

To maximize impact, you’ll want your generative AI to answer the most number of customer questions with the least amount of training and effort.

If you have a categorization of the types of questions asked by your customers, start with that, and see what are the most frequent types of questions.

Figure out which of those questions could be easily answered by a generative AI; then figure out the knowledge required to answer those questions. (Often, the most frequently asked customer questions – and their associated answers – will already be organized in a FAQ. Train the AI on the FAQ.)

Iteratively train the AI on other categories of knowledge, starting with highest volume categories first, and working your way down to lower volume categories.

Also, remember, generative AI can do more than just answer questions – it can help customers accomplish self-service tasks, like look up an order status or initiate a return. Understand which use cases/tasks can become self-serve with AI, prioritize those tasks based on volume, and start integrating your generative AI with database APIs to extend your self-service capabilities.

Do I need to collect all support knowledge in one central location?

No, a best-in-class generative AI solution should be able to crawl knowledge from multiple sources, including but not limited to:

• Existing support knowledge bases and FAQs
• Corporate web sites
• Slack channels
• Discussion boards
• Google Drive folders
• Sharepoint directories
• Resolved help desk tickets

Additionally, a best-in-class generative AI solution should be able to ingest knowledge in multiple formats, including (but not limited to):

• HTML
• Word and text documents
• Spreadsheets and CSVs
• PDFs

What if I don’t have a perfect customer service knowledge base?

Many people might be hesitant to implement generative AI because they don’t have one curated and verified source of truth for customer service knowledge. They’ve heard the old adage, “Garbage in, garbage out,” and their instinct is to defer the deployment of generative AI until a fully accurate knowledge base is created.

There is some truth to this statement – i.e., if there’s incorrect or conflicting knowledge in your support knowledge base, and you feed that knowledge to an AI, it is possible that the generative AI will provide a customer with an incorrect answer.

First, note that the generative AI creating incorrect answers is a probabilistic outcome. If the correct answer occurs 99 times in the knowledge base, and the incorrect answer only appears 1 time in the knowledge base, it’s likely the case that the generative AI will produce the correct answer 99% of the time.

Also, let’s flip the logic around:

Which knowledge base(s) are you using to train your human agents?

When your agents don’t know the answer to a customer question, which knowledge base(s) do your human agents search?

Are those knowledge bases(s) perfect?

In other words: if a knowledge base is good enough to train/be a resource for your human agents, it’s good enough for a generative AI.

What should I do if I have conflicting knowledge?

Best-in-class generative AI solutions should ensure that all responses are fully traceable, i.e., each response should be annotated with the knowledge base document(s) that the GenAI used to create the response.

Admins should create a standard battery of typical customer service questions that can be used to test the generative AI solution on a periodic basis.

When an admin encounters a wrong answer, the admin should investigate which documents the GenAI used to create the answer.

If the underlying document is incorrect, the admin should be able to do 1 of 2 things (or both) in a best-in-class generative AI:

• Give the AI feedback – i.e., tell the AI to ignore the incorrect answer, and provide the correct answer in the feedback; and/or
• Correct the source documentation (if possible) and retrain the AI.

How long does it take to train a generative AI?

The time it takes to train a generative AI is directly proportional to the amount of training knowledge – the more training knowledge, the longer it will take.

For typical customer service deployments, initial training should take a few hours – i.e., not a few minutes, nor a few days or weeks.

How long does it take to get trustworthy generative AI responses?

Best-in-class generative AI solutions typically require a week or 2 of feedback before being production ready.

Note, however, that improving a generative AI’s responses isn’t a function of time – it’s a function of 2 things:

• The support knowledge base used to train the AI – if the support knowledge base has many different (incorrect) answers to standard customer service questions, it will take longer to surface those incorrect answers, provide feedback, and get to the point where the AI isn’t generating incorrect responses; and
• The amount of testing/supervision done. AIs don’t improve automatically. They require agents and admins to ask questions, judge responses, and provide feedback. If no one asks the AI any questions over a 2 week period, response quality at the end of the 2 weeks will be the same as the response quality at the start of 2 weeks.

How do I give feedback to a generative AI?

Best-in-class generative AI solutions should allow admins to give feedback in 2 different ways:

• Providing a simple “thumbs up” or “thumbs down” feedback mechanism
• Giving admins the ability to provide qualitative feedback like, “When someone asks this question, be sure to mention X, Y, and Z.”

How long does it take for a generative AI to “learn” from the feedback?

Best-in-class generative AI solutions should learn immediately (or near immediately) after feedback.

How does generative AI deal with changing knowledge?

A best-in-class generative AI solution should deal with changing knowledge in 2 ways:

• It should automatically recrawl knowledge on a periodic basis, and automatically find new knowledge or updated knowledge during those crawls
• It should allow admins to either provide a knowledge cutoff date (i.e., automatically remove knowledge that is over 2 years old) or favor more recent knowledge over older knowledge.

How do I ensure the generative AI maintains high response quality over time?

Once you’re comfortable with the response quality of your generative AI, there are fundamentally 2 ways to maintain excellent response quality:

• Maintain AI knowledge base quality
• Version control the generative AI’s knowledge base
• To the extent possible, curate updates to the knowledge base. If anyone is allowed to make updates to the knowledge base, without any quality control, the quality of the knowledge base (and the generative AI training on the knowledge base) can degrade
• Stage the generative AI. When a generative AI trains on a new, updated version of the knowledge base, don’t ship the generative AI straight to production. Instead, stage it first and assess response quality.
• Create a standard set of “quality assessment” customer questions
• When the generative AI is in staging, ask the AI the standard set of questions.
• Assess response quality
• If response quality has not materially changed, ship the AI to production.
• Add to the quality assessment questions as need be
• Ask for a customer who is using the integration live in production.
• (If your environment is unique) Negotiate a POC to prove that the solution works with your environment.

How much effort does it take to maintain a generative AI solution?

It takes very little effort to maintain a generative AI solution.

The only effort required is around knowledge base management and periodic quality assessment.

A person or a team of people should be responsible for managing the support knowledge base that the generative AI trains on. You may already have this person (or people) already performing this function – if that’s the case, then there’s no incremental effort here.

A person or a team of people should create and update the list of “quality assessment questions.” (See above.) This same person or team of people should also test the response quality of the generative AI, especially each time there is a major update to the AI’s knowledge base.

Both knowledge management, managing the list of quality assessment questions, and assessing the response quality of the AI could be taken on by human agents.

Note: the need to assess ongoing AI response quality can be decreased by using the AI in Agent Assist mode, where the AI drafts customer service responses for agents, agents review and edit the response, and the AI learns from the edits.

How do I handle customer data and privacy when using generative AI?

If you’re concerned about data privacy, look for the following in a commercial generative AI solution:

• Make sure the solution is SOC 2 Type 2 compliant. SOC 2 Type 2 is the SaaS industry standard for data security and privacy. It’s not a guarantee that your data will always be 100% secure and private, but it is a third-party audited stamp of approval that your solution vendor follows best practices to ensure the security and privacy of your data and your customer’s data.
• If consumers submit personally identifiable information (PII) to the AI, make sure (1) the AI can identify PII and (2) it obfuscates the PII before sending it to a sub-processor (like an LLM) or storing it into a database.
• Only connect your generative AI solution to databases that don’t contain PII (like passwords, email addresses, names and addresses). If someone hacks into your generative AI solution, constraining which databases your GenAI connects to limits the blast radius, so to speak.

How do I scale my generative AI solution as my customer base grows?

The great news about generative AI is, it’s infinitely scalable. It can handle an unlimited number of simultaneous customer inquiries as you can imagine.

As your contact center becomes more comfortable with generative AI, you might look to:

• Expand the use of generative AI across channels: Some brands might first look to use generative AI only in asynchronous channels, like email. Brands could expand generative AI use to real-time channels like live chat and voice.
• Expand self-service capabilities: Any database that customers might need access to (read or read/write) can be integrated with a generative AI solution. Customers can then interact with that database via a natural language conversation. Examples of self-service capabilities could include checking on order status, changing an order, canceling an order, initiating a return, checking on backorder status, and being added to a waitlist.
• Create more targeted generative AIs: Some customer segments might need more specific customer service, e.g., VIP customers, customers using a specific edition of your product/service, customers with custom features. It might be useful to create generative AI specifically for those customers/customer segments.

How do I ensure compliance with industry regulations when using generative AI?

Here are a few best practices to ensure compliance with regulations in generative AI:

• Be sure to select a generative AI solution that doesn’t hallucinate. Hallucinations can result in factual errors that may result in compliance violations (e.g., truth in advertising, health and efficacy claims, etc.).
• Ensure your generative AI vendor maintains compliance with the regulations that apply to your company (e.g., HIPAA, PCI, GDPR, FERPA, etc.)
• Ensure your knowledge base complies with all applicable regulations
• Ensure your generative AI solution is SOC 2 Type 2 compliant. Your generative AI solution might be integrated with sensitive parts of your infrastructure. For example, in a famous hacking incident, a ticketing event vendor (who was PCI compliant) had a chatbot installed on its website. The chatbot was hacked into, and the chatbot became a defacto credit card skimmer for the hackers.
• Provide your generative AI with proper guidelines. For example, if your AI provides medical advice, you could provide a guideline, “Always mention that this is not a medical diagnosis and recommend a visit to a licensed medical professional. In case of a life threatening emergency, dial 911.”