Welcome to No Jitter’s Conversations in Collaboration. In this current series, we’re asking industry leaders to talk about how AI can boost productivity – and how they define what productivity even is – with the goal of helping those charged with evaluating and/or implementing Gen AI to gain a better sense of which technologies will best meet the needs of their organizations and customers.
In this conversation we spoke with Jonathan Rosenberg, Chief Technology Officer and Head of AI at Five9. Prior to Five9, Rosenberg was at Cisco, where he was CTO for the Collaboration Technology Group (CTG). He is also known for his authorship of the SIP protocol, which is the foundation for modern IP-based telecommunications.
In our conversation, Rosenberg suggested that Gen AI solutions in the contact center should be viewed through the lens of their potential impact on the customer experience and added that any AI technology should be assessed by how it applies to existing problems.
NJ: Some CCaaS vendors provide access to multiple large language models (LLMs) while some have “standardized” on one, at least for now. How might an enterprise…a contact center…go about evaluating a generative AI solution?
Jonathan Rosenberg (JR): This technology is so new, so bleeding edge and it’s evolving at such a breathtaking pace that our philosophy [at Five9] is – has always been – that we need to use whatever [model] is best in class and allow either our contact center vendors to pick the right ones [for them] or our implementation teams will choose the right tools for the right job. These [models] have different cost and performance tradeoffs, they're evolving quickly, and so what was the best or cheapest yesterday is possibly not the one that's best or cheapest tomorrow. You have to be nimble and leverage the right tool for the right job.
NJ: You’ve mentioned cost, performance, and implied different capabilities among LLMs. How might an IT staffer go about evaluating one model versus another – or even compare it against the solutions already implemented in their contact center?
JR: Most people don't pick what flight they're going to take based who makes the jet engine. They care about the end-to-end experience, which is the flight. That's the basic analogy we use. You have to look at the use case you want to solve for.
At the end of the day, the enterprise contact center manager should not be thinking about this as if an LLM were an ingredient you’d use – or vendors like us use – to build a solution that delivers value. [That solution might be] a high performance chatbot or an agent assist capability or call summarization. That [solution] is what they're picking and that's the thing they must compare with what they're doing today.
NJ: You use OpenAI – is that direct into OpenAI or is it through Azure? And what’s your take on using multiple LLMs? I’ve seen that Microsoft is adding other foundation models to its Azure portfolio.
JR: We currently use [OpenAI] through Azure. As [Microsoft] adds models, we will make use of them. At Five9 we have an AI team who constantly evaluate these different models to understand their cost and performance, both OpenAI’s and [models from] others. As we see good results, we add them to the repertoire of solutions that we can use with our customers.
NJ: You mentioned call summarization – that’s obviously one of the prominent Gen AI use cases in the contact center. When I spoke with Christina McAlister with Forrester, she mentioned that there might be a cost issue if you’re hitting the large language model on every turn of the conversation versus just sending the entire transcript out and getting a summary of that. Do you have any thoughts around that? What is Five9 is doing?
JR: This is where there's different concerns for the contact center manager than there are for the vendor who builds the product. The contact center manager – the contact center buyer – cares about the price of the product…how much summarization costs, for example. At Five9 we have a price book for this and other [use cases]. We’re not just reselling the per token cost of the AI model to [the customer]. We sell our product with per agent prices, as we price most of our products. We worry about that price being good value for the benefit the customer is getting. That's what the contact center manager should evaluate.
Now, there's a different question, which is: Okay, under the hood, how does your tech work? Does it engage with OpenAI continuously or does it send [data] at the end of the call? Different use cases involve different ways you'd use it for post call summarization. We do it post call, [so] we take the whole transcript and [use it] to produce a summary.
What we do, how we use the LLM and when we send data over, how we jiggle the bits around is our problem to solve for the contact center manager. That’s what we worry about. We focus on delivering the business outcome, which [in this example] is high quality post call summarization.
[To reiterate], the manager of the contact center isn't directly buying the LLM and isn't the one directly making the decisions about how to send transcripts to the LLM. That's what vendors like us do. We make those decisions. And then we sell the product to the contact center manager. The contact center manager then has to decide: Okay, am I choosing Five9 or one of [their] competitors? Does Five9 have post-call summarization? And how much does it cost?
NJ: All of these LLMs are deployed in the cloud. And the contact center market overall, as you’re well aware, is a mix of on-premises versus in the cloud. If you're on-prem only is there a way to get Gen AI in your contact center?
JR: No. I mean, the last gasps of prem are really gone – who can you even buy anymore? [Most] are on maintenance contracts that are running out. Five9 was always cloud while our competitors have had a mix [of on-prem and cloud]. And with AI, you take that mic and drop it. You have to go to the cloud for [Gen] AI; it's impossible to do it on-prem.
NJ: Is Gen AI that final impetus to get people from on-prem to cloud?
JR: I think the push already happened. [AI] is further helping people with a little extra oomph to make that migration to the cloud, but [the trend] has been in full swing for a while.
NJ: In some of your other interviews, you’ve mentioned that the summary of a conversation is presented to the agent and then they check through it. Is that basically where ‘human in the loop’ is at with Five9 right now?
JR: For that particular feature, yes. That's one of the reasons [summarization] is the first use case Gen AI has brought to the contact center. Our competitors are all doing it. In one of your previous interviews you talked about it there, too. Summarization is a really good ‘no brainer’ use case [because] it has the ‘human in the loop,’ a very low risk of harming customer experience, very easy ROI, and it’s something every contact center manager [can understand how] it can reduce call handle time. It requires no explanation. That's the first step. There's a lot more to do and we'll see increasing applications of use cases.
What's going to happen is we'll see a range of different guardrails – some that have ‘human in the loop’ and some that don't, evolving the management of how much (or how little) data it has access to, how much you ask it to do and say and how much you don't, and how often you put it in front of the customer – as well as adding monitoring, testing, and oversight, like you would for a person.
I almost think we'll see [quality management] QM for AI where it’s a ‘who watches the watchers’ approach. We'll have people making sure that the AI isn’t saying inappropriate things and then adjusting it via the controls that are in place. This is active work we're doing in Five9, and it's going to be essential for bringing this technology safely and responsibly into the industry.
NJ: So how might that work? Would that be Five9 making those alterations, or might it be individuals in contact center flagging those scenarios and making those changes?
JR: We're going to provide a set of the tools that allow the contact center manager to build, test, train and monitor the generative AI use cases in the contact center – and make it easy for them to do that. We think that's a whole new category of technology that we'll have to put into the hands of the contact center manager. [Historically,] we put other tools in their hands – IVR builders, campaign managers and QM, which is a close analogy to [Gen AI] because QM is all about managing quality and defining what you want to have happen and then making sure it happens.
NJ: I see the terms grounding, training, and retrieval augmented generation (RAG) as ways to take a big model and ‘ground’ it in the corpus of your particular enterprise.
JR: That's exactly what the tool I'm talking about will do. It’s controlling the input, the data that the LLM has access to. That is one of the big ways in which you manage the large language model is you tell it what you want to do, you give it the data you want it to operate on and you tell it to only make decisions based on this data I just gave you.
Under the hood, there's cost because most of the model vendors charge based on tokens and these [different use cases] all change the amount of tokens that get consumed. It's our job to simplify that.
We don’t charge our customers per token. Nobody knows the value of the token. How much do you think a token is worth? You can't quantify that. But you can quantify and understand the value of [something like]: ‘This will save me X amount of minutes per agent per month’ or ‘This will improve my CX score, and it's going to cost me X dollars per agent.’
Those are things that people can understand, so that's the way we’re pricing, and we'll worry about token costs.
NJ: When your customers come to you, are they looking at that saving time per call, or to cut costs or are they looking more at the Gen AI solution itself?
JR: It’s all so new that customers don't exactly know what they want. Part of what we do, which is a big part of our values, is to be high touch with our customers in both pre and post sales and help them understand the value of the product. We have tools to help [them] understand ROI, and we go in and explain the benefit and what they can expect to see. What we've also done is let customers try it before they buy it.
Historically, this is unheard of in the contact center industry. Other businesses do it all the time – ‘try before you buy’ is not a new idea. You couldn't do it on premise, but you can do it in the cloud. Even in the cloud it used to be prohibitive because there was such a high implementation cost that it just wasn't cost effective. But now with these [newer] technologies, we can do it. So, we're letting customers try summarization and judge for themselves whether or not it's giving them value. If they like it, they purchase. And they've been purchasing – I’m pretty happy with those results.
NJ: As they roll it out are they conducting AB tests?
JR: It's been a mix. We've had some of early customers be diligent where they had agents taking [call] notes and then they’d put together a spreadsheet [to track] ‘this is what the agent wrote down’ and ‘this is what the LLM wrote down.’ Then they compared them to get an accuracy metric. On ROI, we've had some of them look at changes in handle time.
But it isn't just about the handle time and that's one thing I think the industry has been obsessed about for too long. The focus needs to be more about customer experience. [Summarization] actually helps improve customer experience.
You’ve probably had an experience where you call the contact center, explained the issue you’re having, and the first agent is not able to resolve your issue. So, you call back and have to explain the whole thing to the next agent. It’s awful, right? Part of the reason for that is, historically, it’s been too expensive to have good notes – in a brief and concise format – about what happened on the prior call that allowed the next agent to quickly know what happened. Well, now we can do that. Pervasive deployment of this technology will hopefully elevate the industry.
NJ: Gen AI can be deployed pre call with virtual agents, but you already have non generative AI approaches to that – the ‘traditional’ chat-/voice-bot. Does Gen AI represent a wholesale replacement to that model or is it best used to help with intents and entities?
JR: That is a fantastic question. And it's one where different vendors are taking different approaches. It's our belief that there's a continuum. We refer to this idea as the ‘dial of trust.’ It gets back to managing the risk associated with Gen AI saying arbitrary things. ‘Human in the loop’ works great for agent assist, but it doesn't work for voice bot or chatbot – that’s the whole idea…no humans. So how do we manage it?
One of the ways you manage it is this continuum where at one extreme there's no Gen AI at all. That’s the current intent and entity extraction technology set. But then what you can do is add more and more. You can just use Gen AI for intent detection and entity extraction, but not to generate text that is a sent to a human. That mostly reduces the implementation costs but doesn't necessarily do a lot for the customer experience.
Then you could say, maybe what I'll do is use Gen AI for part of the knowledge inquiry – a customer asked a question, so I'll use Gen AI to answer the question, but I won't use it to execute a self-service process because I don't trust the AI with API access yet. And then maybe there are use cases where you do want to do a little with APIs, but [control how the AI uses them].
What we're building – and this gets back to our core philosophy of giving customers tools to manage all this – are tools to integrate Gen AI into a classic, directed conversation flow builder that will allow you to pick where you want to dial that knob based on your business needs. That tooling will be essential for bringing this technology to the contact center.
There will be other vendors doing that. If you throw a rock in Silicon Valley, you'll hit a CEO of a contact center Gen AI start-up. [Their approach] is, basically, ‘Forget this old-fashioned way of doing it, I’m just going to use Gen AI.’ That is all great demoware. In some use cases, that’ll be fine, but for most contact centers, they're going to want different use cases on different operating points on this trust spectrum. And the vendors that win are ones like us who will provide that continuum with a unified tool chain that lets you take the best of the old and the new and put them together. That's what we're doing here.
Want to know more?
Check out the following for:
- Five9's privacy policy, which discusses how they process data that they control.
- Five9's data processing addendums (DPAs), which discuss how they process customer data.
- All other legal documentation can be found here.
Also check out Five9’s recent launch of its GenAI Studio, as discussed by frequent NJ contributor Zeus Kerravala and in NJ’s coverage of Enterprise Connect 2024.
