Transcript for "Deploying conversational AI: Implementation do's and don’ts": Okay. Let's get started. Welcome to our webinar, conversational AI, implementation best practices to Don't. Presented by Donna Fluss, president of DMG Consulting, and Mark Langanki, president of AI Implementations for IntelePeer. I'm your host, Paula Rivera. Today, we're diving into a topic that's transforming how businesses engage with their customers, conversational AI. While the potential is enormous, successful implementation is far from guaranteed. I'm sure a lot of you can relate. That's why we've been brought together two incredible thought leaders to help us navigate this space. Combined, Donna and Mark bring years of customer experience, AI strategy, and technology expertise to the table. Our conversation will be free flowing today and insightful. We'll explore everything from the evolution of IVR to IVA, the importance of agentic reasoning, how to align business goals with the technology, and what it really takes to drive adoption and measure success. Today's webinar is being recorded and will be available on demand over the next few days. You'll get a link to the session via email once it's available. Also, we want to make sure you get everything you need out of the webinar and encourage you to submit your questions throughout the session. We'll address those at the end of the call. You submit them in the q and a box, but if you put them in chat, we're good with that as well. So please do submit questions. So let's get started and uncover what it really takes to deploy conversational AI the right way. Donna, Mark, welcome. Thank you. Good morning or afternoon. Great to have you with us. We're gonna kick things off by grounding ourselves in the evolution of voice technology. We've come a long way from traditional IVRs to today's intelligent virtual agents. But what does that shift really mean for business and customers? Mark, let's start with you. Yeah. Excellent. So, you know, when I think back of, you know, kind of our IVR days, I'll call it, you know, that that probably span about thirty years, we only had the opportunity to really do four things. And that's just all the technology allowed us to do. And it kinda got our mindset into, you know, the concept of press 1 for sales, 2 for service, and 0 for getting to the agent, because that's all I could do. And we we started to dabble in at what I would call the second phase or the initial phase of IVAs, which was really a different form of AI, conversational AI, which was really kind of focused in a form of AI of that that's really pattern matching. Right? Did did if I, you know, said something that sounded like ask for a nurse, did I did I get that right, and did I go down that path? It was still very IVR flow ish, so it wasn't really as as human like or as logical as you'd think. But when the generative AI came out, part of what we started to do was try to make generative AI work as an IVR and and still make that flow. But the reality is is generative AI, by its term generated generates output. And to me, that's really the biggest, you know, piece of what is new for us today. And although we're we're finding a lot of the use cases are replacing IVRs or getting in front of, what was mostly inbound. Now we're finding about an equal inbound and outbound capability of doing communication. But now in this generative AI world, with voice, text, however you wanna make these communications work, It's opened up a tremendous amount of new ways of, you know, looking at how can we augment, you know, front desk workers or, you know, in intake, people. And and kinda taking away from things that would never have been, you know, a, in a contact center, but, b, also, something that, you know, as time progresses, we're finding new and new use cases that we never thought of before because now we have this ability to generate this, you know, output to customers and have these interactions, that we can only do with humans before. So it's not just the technology at this point. It's the technology of how can we emulate and do things, with our business that that maybe we couldn't have done before. I I love it. Donna, what what say you? What are your thoughts on this subject? Well, to begin with, I agree with everything that Mark said. And and, Mark, actually, the first VRUs that I played was was actually back in the nineteen eighties. And I had just taken over what turned into be a very, very large call center, although it wasn't at the time. My boss came up to me and said, Donna, your VRU is down. And I said, okay. If tell me what it is and where it is, we'll figure out how to put it up. But, you know, the world has really transitioned and changed over these over these few years, and you really pointed it out well. You know, we had this whole history, IVRs, and they did, as you said, four things. They were matrices, essentially. Right? They were four by fours on top of everything else, and that's what they could do. When we talk about GenAI enabled IVAs, intelligent virtual assistants, or conversational AI solutions, which is, again, another way of saying GenAI enabled IVAs, it this is not an upgrade. Right? So going from directed speech recognition to natural language speech recognition, that was an upgrade. That was a, you know, a a small change in technology. When we go to GenAI enabled and anything GenAI enabled and particularly the self-service solutions, you're talking about very different technologies. So this is a whole transformation. And, you know, as you were pointing out, what was unimaginable two years ago is amazingly now possible. And what will be possible I mean, it gives me the goose pimples as I think about this. And what will be possible two years from now, you know, from a conversational AI perspective and many other AI perspectives. We can't even we're not even imagining it. So, you know, the rules are are gone. And I think, you know, what's really important, you know, when we think about where all this is going, in particular, you know, conversational AI, these self-service capabilities, please, you know, leave the past behind and think about the future. Well said. So I really appreciate what both of you are saying. If you know, to the neophyte, one kind of sort of clumps all the AI together, generative, conversational, and whatnot. But it really sounds like the advent of generative AI enabled for the two way natural conversation that we have today that, again, as Donna, you just said, we didn't really have two years ago, three years ago. So I think that's really super interesting. Mark, were you gonna say something? I may have cut you off. No. I I I haven't heard Donna talk about it that way, but it it is extremely important that, you know, the the thing I took from that was, you know, let let's not draw from the past, but also, you know, just think what what is possible, what could we do with this because a lot of what we've been after in, you know, the re recent six to twelve months, we never thought of before. We just didn't have the technology to do it. Excellent. We're gonna dive on into what our Fred is the pundits are saying. So there's no shortage of opinions, obviously, in the AI space. Good, bad, indifferent. But if you look at some of the data, it is eye opening. There was a recent MIT study that suggests 95 of AI implementations are failing. But is that really the case? We're going to unpack it that unpack it that find I find that number absolutely hard to believe because if that were the case, why would people adopt AI? Donna, I know you have strong opinions on this, so I am going to start with yourself. Yeah. Absolutely, Paula. And this one really does drive me crazy. We were talking about it beforehand. And I generally like this kind of, if I may call them, pop up studies. But this one, while it got a lot of press, is really very misleading. You know, yesterday, I was reading The Economist, and it said that 15% of AI initiatives are are succeeding. But let's unpack the, MIT study a little bit. First of all, kudos to the marketing person who picked that one line, that one headline that got more attention than, you know, they will probably get throughout the rest of their their their professional career. You know? So let's start with that. But but let's go a little bit deeper. First of all, when you read through it, it points out a lot of different things. So it actually does share more information. To begin with, fifth this study was based on 52 surveys. It wasn't geographically relevant. It wasn't vertically relevant. Whoever did it reached out and got 52 people, and from the way the study is written in the areas of sales and marketing to, answer their questions. And as, you know, Mark and I will probably both be talking about throughout throughout today, you know, how AI gets rolled out and who rolls it out, does vary from company to company. But a lot of cases, it is department specific, so it's not overall. The study does point out that if you look deeper into the area of CX, which is classically customer service and contact center, that those threatened back office, AI initiatives are working better. And I'm gonna tell you without a fact that they are that they are. They're extremely effective. What's also interesting about the study is that it's really about agentic AI. Alrighty? It meant that, it's really about actually generative AI, and it mentions agentic AI at the end. But this is really a study about, generative AI. And from the get go, there were a lot of misconceptions about what generative AI, which is very powerful, is absolutely a missing link in the world of CX, but its purpose was not to eliminate all agents. And so, you know, if that's what you're going into something thinking you're going to achieve, that was never gonna be the case. Now I can go on and on about this. But, Mark, what do you think? Yeah. We we gotta be careful on timing here. We'll talk about this for the whole time. But, yeah, I I would say, first of all, us at IntelePeer are seeing the exact we said a 100. But, you know, again, when you start to rip that that apart, you have to also think about, well, what generative AI are you talking about? Because I kinda classify it as you have personal, my co pilot. You have, you know, productivity or, you know, how do how do I create enhancements, and then you have kind of the enterprise view of things. And, you know, you also read through their, a little thread if you followed it was they talked a lot about proof of concepts. I don't know about you, but how many proof of concepts have you ever seen that were successful anyways, even with, you know, tried and true things. So so the reality of it is I I I took, you know, half a grain of salt with some of it. But, you know, there were a couple pieces in there that I thought were somewhat interesting and and, you know, I'll talk about those because we have a couple other topics to talk about here. But, you know, the the reality is is that AI by itself is not a thing. You know, it's not a physical thing. I mean, the you buy it and, you know, pay for it and all that. But it has to be integrated with something else. It has to be part of a business process. It has to be adopted and and have the right people. So I do agree with part of it that, you know, obviously, when you have a proof of concept, if you don't have buy in from the stakeholders and you don't have the right IT folks and you don't have the ability to get stuff done, you're gonna fail anyways. So if you took that off and took those other 52 people and said, okay, how many production, you know, voice, you know, AI related apps are out there working, I think you get a different number. But we're we're seeing that we're seeing that, tried and true, you know, in, you know, high nineties versus, 5%. Excellent. And and one of our audience members just chugged in, Very excited. That right there, Mark, that's the failing I see. So so there was agreement amongst our audience members with what you're seeing, Mark. I appreciate that. So let's dive in a little bit into agentic reasoning. It's a term that's gaining traction in AI circles, but, like, what does that really mean? It's not just about automation. It's about enabling systems to make decisions. But how do we design for that? And, Mark, you've talked extensively about this, so I'd love to get start off with your thoughts. Yeah. Well, it and it is interesting when you, you know, Agent Force was kinda one of the first that came out and that got, you know, the buzz going around. And and I think everybody, you know, mistakenly thought that agentic meant that you have these little teeny agent bots that go, you know, and you have millions of them, and they all work with each other, and they all do this stuff, which isn't true. Agency or agentic actually comes from the concept of agency, and I should pull out my Webster dictionary here. But, it it it basically means to give a human the ability to do their own reasoning and act on or behalf of somebody else. You know, you think of a real estate agent, what is their job? They're out there to make sure that all the things that go on in that transaction are good for me, the buyer, me, the seller, because I don't know how it works. So that's exactly how you think of this. Agency, when it comes to AI, is really around how do you put, you know, a single prompt, not to get too nerdy here, but a single prompt around something where it knows everything about what it's there to do, what data it has access to, how it should think and feel and talk, how empathetic it should be. All the pieces that go into that is really where agentic takes off versus it being kind of this worker bot that does stuff. And that I think goes back to the RPA days. But the true agentic model around that really is being able to have that free form. And when it has enough of that context, that's where this human, you know, ability to have a conversation with it and ask it questions that you'd never thought you'd ask it, or or give it a definition of something, you know you know, impossible. One of the ones we did, you know, probably about six months ago now, is, we were testing different ways of doing birth dates. And so we gave it all the stuff about birth dates. And, they you know, I I did a test call and it said, oh, what's your birthday? And I said, well, it's the first month of the year. It's, two times seven is the day, and it's the, year after the UNIX epoch. So 01/14/1971. And it got it because it thought about it and was able to reason around that because they had enough information. I love that. And, honestly, you just sent me straight. I probably should not admit that, but I really appreciate your your your explanation. I was one of the ones thinking, oh, it's a bunch of you know, it's a 100 little agents running around doing things. So I appreciate that. Donna, what do you what how would you describe it? I think Mark really touched on a lot of what's happening in the marketplace as it relates to different technologies that that come out. Right? So generative AI, November 2022, the hottest thing. Then, although, by the way, Generate AI had been around for decades already. You know? Then all of a sudden, a a Gentic AI. Oh. And as Mark Langanki said, you know, you kinda think about the the bots come marching, you know, two by two. And but, again, that's not what's happening with with these technologies, although, there's a reason that there's a lot of confusion in the marketplace. And what I'm also gonna put out there, is that, the real winner, in a lot of cases, is not just gonna be one AI technology. It's gonna be a combination of AI technologies working together and even if they are working going two by two, but that's not what they're doing. In, you know, in the world of conversational AI and particularly self-service, right, it's Gen AI's ability to understand intent and Adjenic AI's ability to know what to do with that as is the example. Now one of the challenges, though, that Mark Langanki alluded to in the market is that a lot of times, technologies come out, and we're not really sure what they mean. And there's a good reason for that. It's because every definition is different. Now, my firm, DMG Consulting, is a CX industry analyst and consulting firm. And one of the things we do when there is a lot of confusion is that we'll write a definition to act as a lightning rod, not because we think it's the right definition or we're not gonna put a wrong definition. But the idea is to put something out there so everyone can tell us what's wrong about it, which is what they do, as well as what's right. So just a little bit of of on this definition, and it's gonna pull together what what Mark Langanki was saying. Right? So AgendaK AI, to begin with, is an intelligent development framework. So it's not little bots, but it's an overall development framework. Okay? So Agendik AI agents are engineered and built to perceive, identify goals, reason, steer workflows, and act independently across technical and multimodal environments. I mean, this is really a very broad capability. IdenticAI has agency. Right? And it's that agency that's so very important, and it enables them to autonomously set goals, make decisions, take actions, adapt, and self improve in real time. Now there's room for improvement in these areas. What we have today it would be nothing compared to what it is we're gonna have a year from now, two years from now because the development of these technologies is also happening in AI time. So it used to take twenty years. It's now taking, Mark Langanki, what, at most? Nine months, a year? Oh, yeah. And it it's amazing the the amount of time cycles that are going down now. And and and it's not just the voice and and, you know, the agent piece. I mean, you know, even how we develop now, you know, it it it went from one developer because that was you can only have one person on a keyboard with a screen because you didn't have networks. And then you had teams of people, but now with Vibe coding and getting that down, it's back to one. And I've seen things, like, you you put all stuff together and boom, you know, in twenty minutes, you you have frameworks of of, you know, full, you know, lamp stack to, you know, all the way out applications running that I could never dreamed of building within a year. Yeah. I I concur and I I appreciate the fact that both of you killed my vision of little bots running around doing things for me. But but I love it and I appreciate it. Let's start getting a little bit more into the meat of, you know, what does it take to to get to the success stage? How do we align business goals with AI design? And how does that lead to better outcomes for both companies and their customers? That's extremely important. Donna, what are you seeing as it pertains to to doing that alignment between business goals and design? Boy, did you set this up well? Because it is all about business outcomes and, specifically, return on investment. You know? Again, Mark alluded to this. Excuse me. It's not AI for the sake of AI. It's AI for the value that it brings to the organization and its customers and its employees. Right? So, you know, everyone talks about AI as it's it's the end all be all. No. It's it's a development environment. Right? But you've gotta make sure that what you're doing is very, very specific. Right? So you have a strategic goal within your organization. And then how do you address that goal? Right? And this is the case regardless of the programming language that you use. And, of course, AI is a programming language, a very sophisticated one, a transformative one, but it's a programming language. Right? So companies need to first come up with the right use cases. Once they come up with the appropriate use cases, then they should find the technology to help them drive the right outcomes. And in a lot of cases today, that is gonna be AI. Then, right, you wanna have the ability to measure, what's happening. So then you have to come up with the right KPIs, key performance indicator metrics in order to measure. And before you start any of this any of this. Right? I mean, you know, you you come up with your use cases and what outcomes. But before you make any changes, you want a baseline if you have something from the past, what what it is so you would know the effectiveness. And if, as Mark Langanki also mentioned, a lot of what's going on right now with these new technologies is that companies are able to tackle, outcome outcomes and opportunities that they weren't able to address with our really, highly structured, you know, IVR technologies. And so so if you run into one of those situations, there's a lot of them, what you wanna do there is you want to, again, know your outcome, your use cases, your outcomes, your KPIs, have your reporting in place, right, so that you can show, you know, as each month passes, how much better you're doing. So what I'm saying is that regardless of project, it's a project. You've gotta follow the right best practices, and that includes being very, very, very, very careful about what it is you're defining so that you can track it and measure the results. Mark Langanki? Yeah. And I I will have to go back to the the MIT study because we we poured over that one. But there was one, sentence in there that I totally agree with that makes a lot of sense, with this, which is we have to at least in IntelePeer, we have to act as a BPO, not as a SaaS company. And what that means is that we're going to go as far into our our customer's business as we need to, which hasn't really ever been the case in the past. You know, when, you know, if you think of the early days of contact center, you know, how do you want your calls to route? Well, press 1 for this and 2 for that, wait a while, and then your call is really important. We're recording it blah blah blah. Yeah. That's all we had. So that's what you did. And it didn't matter if you were routing into, you know, a, you know, car delivery, company or a, you know, manufacturing company. It was just how do you wanna route it. Now that we have to think about how the AI has to be agentic and think about this, we have to be very, very intimate with understanding how customers, you know, whether whether it's, you know, our customers to their patients or to their customers, how we make that interaction work well. And a lot of it is still that newness we're talking about, Donna, that, you know, pressing 1 for sales and 2 for service. No. No. No. No. No. That that that's not it. There's the intent upfront, but then it's really where does it go from there. I'll give you a a quick snippet of one of our customers, one of my favorite, Domino's. I didn't I I I worked at Pizza Hut when I was in college. Yes. So I I know a little bit about the trade, but I have I I never would have Donna on me that I know as much about Domino's as I do. 22,000,000 ways to order pizzas, stores can't or delivery can't be over $50 because they can't carry that, you know, that in cash and, you know, I mean, just it goes on it. Garlic is the default on the crust and you can't get a square cut on a New York and Idaho. I mean, it's like, that one I don't know is true, but, something like that. And but you have ingredients list that are all over okra in one part of the country and, you know, Arizona's gonna have their their peppers and, I mean, all all the different things go into it. And that can't be missed by AI. And a human can be successful with it. I don't know how many upsells or or things I got right at Pizza Hut my day, but the AI, we have to get right. And so we have to think about how do we build an agentic framework around that experience because that is driving the business now. But that also gives us the ability to think about, okay, what else could we do? How could we interact differently with customers, patients, and and and the like? But but it is kind of that it's a change of of how much we have to really understand. And believe me, every customer we have is radically different even though, you know, we specialize in in dental, we specialize in everything in health care. I haven't seen two customers anywhere near each other. You know? We have we have one with or two customers with almost the exact same name, and both of them do everything exactly opposite. So it's really up to us as the implementer to really understand the business. And we'll talk about KPIs in a second, but those are the most critical piece of all of this. But you can't have a KPI if you haven't built it correctly in the first place. Well, now I'm thinking about robots and pizza and robots making pizza. But I love what you're saying. I completely concur. It it's no longer just doing the surface level. It's diving deep into what exactly is the overall objective of the business. With that, we're gonna kind of jump into talking about deployment. And deployment deploying conversational AI isn't just plug and play. That's actually a term I absolutely hate because there's very few things in life that are truly plug and play. And I think when you look at AI in general, conversational AI as we're discussing today, plug and play does not apply here. So let's talk a little bit about the real world challenges and process changes companies face during implementation. Mark, I'm going to start with you. I'm inclined to say you see this every day with our customers. Yes. I do. Monday through Friday and on the weekends too. Yeah. So I'll I'll just kinda dovetail off what I was just talking about because as as we, you know, start with customers, it's the age old, you know, you saw a demo that was awesome. It did everything you wanted to do. It sounded well. You know, we kinda joke that that most most of, AI is valued on three principles. What did the voice sound like? Was there latency? And did it say the right thing? Well, that's great for demo, but that has nothing to do with your business. And when we have to scale and get to the right thing, we we we wanna kinda really start with at the kickoff, and that's what we start with is design is how do you run your business and and what's important? What are the things that, you know, we can make a difference with with AI more than just saying, okay, here's here's some technology and let let's see what it can do for you. The merging of, you know, the AI as a communication, enhancer, I guess, you know, because we have the channels already. But but the AI being the thing that's reasoning and and thinking about, okay, well, what what is this patient really after? Are they asking for, you know, triage? Are they asking for, you know, an appointment? And and how do we take that is really, you know, kinda how we start. But the the the reason why we're so successful is is keeping our eye on the entire life cycle. And the life cycle isn't it isn't one life cycle. It's one that iterates over time. But it's also the fact that, you know, our com our customers are implementing this, but it's really based on their patients or their customers that are using it. So how do we really look at that holistically and understand, you know, where things are going and it's even how we sell and and how we support our customers is that, you know, we have an ongoing monthly charge because I haven't seen a customer yet that doesn't have more use cases or new ways of doing things or optimizations or, you know, hey, we noticed that this is happening over here. Can we address this in a different way? And, you know, we have a lot of, tools in our tool bench as well. So we might take somebody that's on voice today, and it's not really not working well. Let's change it to SMS, at least for maybe a demographic or, you know, somebody that has an appointment. And so there's a lot of ways we can move this stuff around. But it's it's it's understanding that in the past when you deployed software, you built something and tested it to be perfect. And then unless something environmental happened or you unfortunately chose Windows as your server, that's a Mark joke, you know, and it goes down, that you don't have to do anything with it. With AI, it's a continual cycle. AI can learn on itself, but more importantly, we learn about how to adapt a a AI and and, you know, our our full system across, you know, our our customer's business. Yeah. I'm gonna I'm gonna jump in for a moment. One one of our one of our viewers just said, you you just absolutely described success for patients in health care, yet 80% of health care cannot define that life cycle and that journey. So, yeah, I think you hit the nail on the head, Mark, which we so appreciate. And I I personally like the fact that, you know, not only are you iterating, but over the course of the life cycle, things change, so you need to be flexible. Totally appreciate that. Donna, what are you seeing? Well, you know, again, Mark touches really touched on a lot of the key points. And I'm gonna first bring this up, and then I'm gonna actually bring it down a little bit. You You know, when it when it comes to any technology, but particularly, things like conversational AI, you absolutely need to spend a lot of time making sure people do picking the right technology. But what drives me mad, to be honest, is that they don't pay attention to the implementation. And as Mark Langanki was pointing out, it's not just a matter of finding somebody who, you know, can say generative AI. I mean, you need somebody with experience in a particular vertical, and essentially in the use case. I mean, it that is the one of the best practices that has been the case with pretty much anything without a doubt, these technologies. You know, generative AI is a just a phenomenal technology for which, you know, we're only beginning to leverage its power, understand its power, obviously, in conjunction with the appropriate LLMs. Right? But, you know, if you wanna have some fun sometime, just just move a comma in a prompt, and you're gonna see a very different answer. I mean, I think generated AI has a sense of humor and gets bored. I mean, I I because, otherwise, I can't begin to explain why it is that certain things, changed. Now from an overall perspective, right, you know, an implementation, right, is is is critical. Right? Technology, but that implementation is is critical. And, I just kinda wanna drill into a couple things that Mark Langanki alluded to. You know, this is, you know so, again, this is not starting from the beginning, a a new phase of an IVR. Right? It's a new technology. And you've gotta have resources who actually know how to use the technology. Now making it even more complicated is some of what's going on now, again, as Mark Langanki pointed out, is not, just replacements of existing capabilities. The power of this is amazing. And so conversational AI technologies are being applied to both new inbound and outbound use cases, for the benefit of the customer, the enterprise, and the agents as as a by the way. But just a, you know, couple of things which we have learned already as an industry is please, folks, when it when it comes to this, start small with one or two things or one or two outcomes that you wanna achieve. And, you know, that is one of the most important practices that we've seen. Another thing that is really, really, really important with this technology is to come up with the use cases, but use cases that have high volumes. Right? So going back to the study, one of the issues is that the pilots are for teeny weeny little things. And if it's a teeny weeny little thing, maybe it worked for Robin Williams. But when in a lot in when he was in the the little lantern. Right? But when it comes to these implementations, it's really important to be able to appreciate, you know, what the potential benefits, really are. There's a lot more we can say about this. The other thing that I just wanna leave you with is, again, something Mark brought up, testing. Yeah. So we all think we are experienced with testing, and and we are. Right? So, classically, there's three types of testing that you're gonna do. That's not the case here. Throw throw out that that that that whole set of rules. You've gotta test, and anytime you move a comma, you've got a retest. Anytime you need to add a new topping, you've got a retest. But and and and along those lines, you're never gonna get it to be the 100% that you want it to be, which is, again, why we get into the whole world of of guardrails and other things like that. But we'll save that for another webinar. Thanks, Mark. I saw you shaking your head. Did you have anything any insights you wanna provide on testing? No. Donna nailed it. I mean, it's you know, it it it we we've entered AI has intro has introduced, the fact that we're not dealing with a, you know, an exact science anymore. I mean, you know, software was pretty much a 100%. I mean, once you get it up and running and you get some of the bugs out, I mean, you know, we we all know and we know of friends that have some systems that have been running for thirty, forty years. Just don't touch it. But with AI, it's AI is kind of a fickle friend. Right? Where, one day, you know, and and we we kinda joke and debate, you know, is it because the GPUs got too hot and they just decided to go go down Third Street on on a conversation or not? But, you know, the guardrails are there, you know, the prompting all needs to be there. But it it the testing is so critically important in various phases, which doesn't just get to how we've we've always kind of traditionally thought of software testing. It testing. It now gets into kind of this quality monitoring thing that we've never really dealt with before. This way, because I I think of the thousands of contact centers I've ever been a part of or helped build, I've only known one that actually recorded and monitored their IVR because it just did what it did. Right? They do agents, but not that. But now we need to, you know, monitor and and record and and understand what's going on with AI because we don't know what we don't know what it's gonna say sometimes. So Excellent. Well, those fickle friends can sometimes irk Donna. So before we do that, actually, it's the users of our fickle friends. Before we do that, we're gonna jump on in and talk a little bit about establishing and measuring KPIs to determine the success of any project as we all know. It needs to be measurable. But the one thing I personally get almost bogged down in or confused by is the fact that there are so many metrics available. So how do you really choose what the right ones are, and how do you tie those to business outcomes? Donna, you mentioned this earlier, so we're gonna kick this off with you. Yeah. This is, this is a big challenge. And by the way, while we we think a lot about it in regard to the AI initiatives, in a lot of cases because, you know, enterprises call me after the fact. I mean, I know I spend a lot of time thinking about this and say, okay. Well, how do we measure the success of our initiative? And I'll say, well, did you baseline? And they'll go like, no. Like, it was a unique thing. And that's pretty standard. Well, a lot of what we're doing with AI gives us gives organizations new opportunities. Some of the best practices really should should still be in place. And so when you're setting up your your conversational AI solution and, for that matter, pretty much any AI initiative, there's certain things you wanna do. Right? You wanna, you know, figure out, as I mentioned, the use cases, the, you know, your expected outcomes. At the same time, as I mentioned, the KPIs and for that matter, the reporting. Right? And, you know, Mark and I have gone back on and forth on some of this stuff, many, many times for a bunch of reasons. But in regards of KPIs, there will be KPIs that are relevant to each of the different things that you're doing. If you're not sure, reach out to the vendor you're working with. Reach out to IntelePeer. Reach out reach out to us. And there are going to be a combination of KPIs that you're gonna want to baseline and measure. So, you know, when we speak about about when we speak about conversational AI, you know, the self-service capabilities, what we're typically organizations are typically hoping to achieve, is either the ability to scale without adding as many people or, right, they're also talking about efficiency gains. And so, you know, obviously, then that's a KPI that you wanna measure. But I'm gonna just caution strongly on on on that one, which is that if you're measuring efficiency gains, please also be sure that you're measuring also a CSAT, customer SAT number, or a you know, which one methodology is NPS because you do not wanna do one at the expense of of, poor service or perception of poor service. Now here's some other KPIs. If you're working in the world of sales and marketing, right, you're gonna wanna look at increased revenue or increased demo volume. But I caution you on that because you could have increased demo volume but decreased sales. Right? So, again, you can see how CSAT kind of kind of fits fits in with that. You're gonna you're gonna wanna look at improvements in handling in the speed of handling and resolving inquiries. You're gonna wanna look at ease of doing business. You're gonna wanna look at increased customer loyalty or, for that matter, decreased customer attrition. You may wanna look at agent retention, speed of onboarding. Right? So you wanna select the right KPIs, for what it is you're doing in the world of conversational AI self-service. Right? You're gonna wanna look, believe it or not, at AHT. Right? So you're gonna need to measure. So okay. So you're displacing more calls, but what does that do to the AHT? Is it good? Is it is it bad? You may wanna look at, you know, FCR. So, again, you know, select the KPIs that are relevant to the particular initiative, not what worked for, you know, corporation x, y, and z, but what works for you, Mark. Yeah. And and to I think that that's a perfect way to kind of think about, you know, the holistic piece of it because, you know, again, you know, when when we only had IVRs and ACDs, we only have what we had. Now when you think about the fact that we're replacing business processes and we're really, you know, emulating a human and what they would do, which by the way, we're usually never monitored, you know, and I mean, we had quality monitoring recording and all that. But, you know, front desk, you know, worker or somebody that was, you know, maybe, you know, dialing for dollars, that that was all just kinda, you know, if it worked or not. And the only thing you could ever really look at was, you know, probably EBITDA and a P and L, which, you know, you'd never tie those together. But, you know, we've kind of stripped this apart, you know, when you get to the technical level, and we really look at it from, you know, I guess, different levels. So, you know, there's kind of a system level of KPIs, you know, are are things up and running because if they're not up and running, what do you what do you expect to go better? You know, and then you get up to kind of an application, which is more the logic of it, and you can look at AI, which is an interesting one now because AI has a bunch of, friends that has to go with it. You have ASR and TTS, so speech recognition and and speaking back. You have different languages. So do we really even understand the utterances that people are saying, and what is it turning into, and how's that changing? So a lot of that is is brand new. And now we we need to monitor and and and measure that because now that's part of the application. But to me, the final one, which is at the top, is user adoption, which, you know, kinda Donna Fluss summarizes, you know, getting to where where where you, left it, is how do we get people to use the system? You know, I I think one of the biggest burdens we have is the annoying, systems that have been created in the past by pretty much everybody, where, the old form of AI would get you into a, you know, oh, did you say, you know, pizza when you said dog and no and, you know, everybody's frustrated. So what's your only recourse? Agent, agent, age operator, customer service, you know. So we see a lot of that, but what we've actually found is we can do new ways of of actually prompting users to when they say that, say, hey, you know, we here you wanna talk to a human. If I could just get your date of birth, we can get this going, and then you actually kind of streamline them into an actual AI conversation. And so we've got various ways that we've been doing that, but that adoption to me is is the critical piece because if you don't have, you know, if you have 10% of your patients, you know, going into scheduling, then you're only gonna get a certain percent of those. And so, you know, we have to ditch some of the containment and some of the, you know, abandonment and some of those, because really what we're trying to do and a lot of what we do at IntelePeer is automation, which maybe if somebody just authenticated, but then they still went to the contact center, let the contact center know they're already authenticated, take a minute, minute and a half off the call. That's still valid. There's still value there. I love that. And that is the perfect segue into our next segment. As much as it is getting the end user to use the technology, the best tech fails if people aren't using it, whether it's internally your own your own employees or or take that one step further, your customers. Let's dive into how to drive adoption and avoid common post rollout mistakes. Mark, it sounds like you have some interesting insights on this, so I'm going to hand the baton to you. Perfect. Perfect. Yeah. One one of the biggest things we've noticed in the last, you know, couple years doing this is that day two used to be where you handed something off to support and all the pro services people said, whoo, we're done. You know, let's, you know, let's go on and do something else. But it's not like that anymore. And so we've kind of readapted into more of a model where we're kinda always working with customers. But now day two is really where the fun begins and everything starts. Because when you're using an IVR and somebody has to press one for this or two for that, they do or they don't. Right? And and I can't train anybody and I can't change it. You know, they're that's I only have the keypad. But in the AI world, when you prompt somebody for something, maybe we find out that a lot of people are, you know, tripping over the the prompt or they're just saying the wrong things or, you know, there's there's things we need to tweak with barge in or sensitivity of of, you know, utterances and of this whole thing. So, you know, we we kinda take a first step on, you know, day two, really kinda stepping back and and listening and learning how, you know, our customers, patients or customers or, you know, students, whatever, are using the system, and then try to identify the ways that we're seeing, people not using it and not adopting. And then we kinda go into these modes of, you know, optimization. But, you know, a lot of times what we do is we split, we split the work into an a b test. So we already know what a does because I was there before. We put in a b and with, you know, the beautiful work all of our, people do with data and analytics here at IntelePeer, we're able to actually see with an a b slider which one's better or worse. So now we're able to, you know, really give our customers a better chance. And then we go to and and people theorize on this. Do you go b c or just go c d? And so we we build all these tests as as we kinda go through there. But we also find and and it seems to be not with every customer, but but it seems to be most of the time we run into some real weird anomalies. Like, you know, one of them, we had was, speaking Hawaiian addresses. Not a language, but and there's a language pack that you can just plug in for it. So we had to go back to the drawing board and say, okay, how do we take the Hawaiian addresses, put them into some form of a reg, and plug it in so it's it's known to the to the AI and to be able to both, you know, hear and speak the line addresses. I I can't give you one because I I tried twenty, fifty times. I can't say them, but, yeah, I can. But but getting that, you know, stuff done, I talked about the date of birth earlier or, you know, the, you know, everybody's favorite, the long alphanumeric, you know, VIN, things. All of those are really where we, you know, kind of really excel and kinda blow blow our competitors out of the water with having the ability to, you know, just keep innovating and keep driving and keep adding new features. Excellent. Donna, how about you? Well, again, it's it's a pleasure to do this with Mark because, I mean, he's living this on a daily basis. And, by the way, I was gonna also use the Hawaiian address as an example because it was such a unique finding, in in the in the overall area. But I think, you know, when it comes to user adoption, I think, you know, the bad news is no more of these, you know, success parties, implementation parties the day after, because it is ongoing. And, really, the name of the game when it comes to these technologies is for it to continuously make itself better and better, and that's very, very important. You know, a couple other things, though, just about, user adoption, conversational AI. If you'd asked me three years ago about, doing an implementation without going through a pilot, I would have shuddered and said, you know, all kinds of things, including, basically, under no circumstances is this gonna be a good idea. But in today's world, it's it's a rather unique kind of situation because if you limit, let's take a a conversational AI to a very small to to to one use case, which you want, but if you limit, its ability to actually practice what it was created to do, you're not really gonna get the overall experience. On the other hand, right, if you don't do that, there are some risks, you know, as associated. So, you know, the the, yeah, the overall adoption, the actual implementation again, and then how it then gets adopted, those worlds have changed and continue to change, which again talks the importance of, you know, working with a vendor who has the appropriate experience in the vertical and doing the kind of implementations that you wanna do. Paula? Excellent. So we are beginning to wind down the the the conversation. A couple of questions have come in. Before we jump jump into the questions, I just wanna do a very rapid fire kinda conclusion and ask for maybe your top three best practices or top three tips for whether it's starting or whether it's kind of implementing, an AI solution. Donna, we'll start with you as as you just have the mic, so we'll keep we'll let you continue on with the mic. Alrighty. So just to build off of what it is we've been talking about, you need to start by selecting the right use cases and outcomes. Because if you don't know what you're trying to achieve, how are you gonna achieve it? Right? You need the right KPIs and reporting to make sure that you can measure it, again, for the same reason. If you don't know what you're measuring, you're not gonna measure it. Oops. Look at this. What does it mean? Right? Another one is please, please, I I'm gonna bang you on this. Start small. Start with one or two things, get it to work, and avoid scope creep. And, boy, is it easy to do scope creep Right. With with you know, it's easy with IVR. But, again but when it comes to Gen AI, well, can't we just the answer is, yes. We can, but let's wait till the next phase. And the last one is the technology is fantastic. It is getting better at a speed we've never even imagined, and it is such exciting times. But please, please keep humans in the loop. Paula? Wonderful. Mark, top three or four tips. My my top three are very easy. It's smart analytics, smart analytics, and smart analytics. So at at a IntelePeer, we have a product called smart analytics that is unbelievably valuable when when used, you know, upfront because what it allows us to do is really understand how your patients or your customers or your students or whatever are actually talking to other people in your business. Because what we do is we we we basically start smart analytics day one, which allows us to both record and transcribe all these calls. Yes. We do all the PII, PCI redaction. But it allows us to understand what people are saying and how they're interacting. Not that we're gonna emulate it, but at least we know how people are Not that we're gonna emulate it, but at least we know how people are willing to have conversations with, you know, this this, you know, you know, dental clinic chain or or whatever it may be. And so turning that on day one is it makes a huge difference. And so turning that on day one is it makes a huge difference in how we think about design and how we actually get into day two and and move along. It's it it can't be underscored and that's why, you know, just like location, location, location for for agents is is a big deal. Smart analytics to us is is, you know, where success starts. Excellent. I I love the plug, I must say. So we're gonna jump into q and a quickly. We only have a few minutes left, but we do have one of our audience members is definitely laser focused on urgency. So I'm going to couple his two questions together. I'm assuming it's a him. He said he asked, please define intent relatively relative to customer urgency. Is that something that can be I don't know if that's defined or measured. But then part two, that was kinda halfway through the webinar. Part two is, is there a use case around understanding customer urgency, and does that develop over time? So define intent relatively relative to customer urgency, and then is there a use case we can discuss? Mark, why don't we start with you? Sure. Yeah. And and both of those, we've been working on for, well over a year now. And and it really kinda goes into, you know, some of the emotions around, like, empathy or, fear or, you know, some of those where if you can tell in somebody's voice that, you know, it's it's an emergency or that they're frustrated or, you know, they they don't wanna have a long conversation, they just need to get something resolved, That can be easily identified with generative AI. And what comes from that is now the ability for us to turn that around and and, you know, as we build, you know, called prompt engineering, as we build our prompts, have the capability to, you know, respond with that empathy, respond with, you know, I'm sorry that that's happened. Or, we we have a, an example where, we we were doing a bunch of testing and the caller talked to the bot, the AI agent, and just was was having a big long conversation. And we we trained it to to, hey, if this person wants to talk, you talk as well. And there was a part of the conversation where, the the AI asked the person, you know, hey. How's your day going? And he's like, oh, it's Thursday. You know, it's kind of the best day of the week. How you doing? And the bot goes off with, well, you know, it's you know, we're really busy here, but, you know, I like it busy and I like to, where did that come from? Because we didn't tell it like that. But because we kinda trained it around that, that that really gave us, the insight to know that that's a capability, that that can grow over time. Now it's it's an evolving piece of the whole pie right now, and it's not something you you necessarily deal with day one. But it is critically important that you get the right tonality, you get the right you know, they're called discourse markers, so the human umms and pauses in there as well. But, yeah, I mean, that's it's definitely something we work on. Donna, anything to add? Just to kind of, you know, bring that up, there's emotion and then the sentiment. And, these are, characteristics, if I may, that, the world of CAI has been really making huge investments to be able to, accurately identify and and respond to. And so already, you know, what can happen now is really much more significant than than it was, you know, eighteen months ago. And, we are seeing just across the board just tremendous innovation in this area because it is so important. Yep. Wonderful. We are coming close to the end of our call. I am going to ask our audience members to keep Donna happy and start small and establish your KPIs. AI is an amazing thing. It can do wonders for your business, and we thank you all so much for participating. Donna and Mark, it's been an absolute pleasure. I've learned a lot. Thank you. Thanks, everybody. Thank you. Bye.