ConfidenceCheck — Your AI Assistant for Error-Free Emails

Speaker A: Well, welcome everyone. With the interest of time today and this being just a shorter webinar, we are going to get right into things, but we're excited and welcome to walk you through Contact Monkey's new feature, Confidence Check. As you guys may already know, with this feature, we are hoping to enable you to take the guesswork out of communication and give your teams that extra layer of assurance before hitting that send button. But before we take you through everything that we've built, why we built it and what it's all about, I did wanna go through a quick bit of housekeeping. If you do have any questions as we go, and I know now you know where the Q&A panel is, my colleague here, Rebecca, from the Customer Success team, she'll collect those and we'll address them towards the end of the session. You'll also see a couple of polls popping in throughout the webinar as well, and we'd love to hear your input there. And of course, as a quick heads up, we will— the webinar will be recorded. So no worries if you want to review it or if you want to share it with the team afterwards. We are going to send it out to everyone after, so no worries there. And with that, I am going to hand it over to Dan to get us started. Thanks for joining everyone.

Speaker B: Awesome. Thank you very much, Keena, and thank you all for joining. We're a big group, which is great to see. I, for one, I'm calling in from beautiful but snowy Toronto today. And I'm very grateful to have the opportunity today to take you through Confidence Check, our new AI-powered editorial assistant. So let's, let's get right into it. And, you know, first things first, why did we build this feature? Well, I mean, this was born out of many conversations that we had with, with many of you. that we had with our users, the recounting of sending an email with an out-of-date or mismatched subject line, or including an incorrect location or link somewhere in your email. This can typically happen when you say, you know, you duplicated last year's holiday hours email to use it as a basis for this year's, and you You modified all the date in 5 places, but naturally there was the 6th place that just kind of snuck by, right? And these are the type of thing that can cause real confusion, especially at scale. And so naturally, you know, that's when that pre-send anxiety can start to creep in, right? You go to send off that email to thousands or tens of thousands of your colleagues, And, you know, as your finger hovers over the send button, those questions pop up. Did I miss anything? Are all my links right? Did I update the dates or the names, the locations in all the places? You know, I'm speaking from experience. I've, I've felt this myself, just sending out mass emails within our organization. And so naturally, you know, we felt that we could do something to help here, to help with this problem. You know, before digging into Confidence Check itself, I'd be remiss if I didn't mention our other existing confidence layers. So if you can imagine, you know, your email proceeding from left to right here as you edit and do your own proofreading, You hit the send button. We have a few layers, you know, in place or coming soon that can, you know, help to correct any issues that you might have. So first off, you know, that moment right after you hit the send button and suddenly then you notice a typo, right? There's actually like, that's a cognitive phenomenon that, that has actually been well researched that You are more capable of finding things when that pressure of sending has gone away. And so deferred sending gives you that grace period on purpose. You can configure a delay of 1, 3, or 5 minutes after hitting send before the actual send starts. So you can cancel if need be. Next up, something that's in the works and should be coming soon is just the ability to cancel sends that have already started sending and landing in recipients' inboxes. So should you need to do that, you can hit Cancel and kind of regroup. And lastly, if an issue with a link has already kind of made it through all of these layers, you know, no worries. After the send has happened, you can go back and you can edit links, and they will update in real time. And this is a feature, very handy one, that we rolled out earlier this year. So what's even better though than correcting an issue after the fact? You know, naturally it's catching, catching an issue before you even send, right? And so that's where Confidence Check comes into play. This is our newest layer in the, the Confidence portfolio. So with, with Confidence Check, You— we pair your email with a carefully designed, thorough set of editorial rules, and we pass it off to a state-of-the-art large language model, AI. And, you know, with that, we have a couple questions in a poll that we're gonna go ahead and launch now that is— they're related to AI usage in your own organizations. So if you take a, take a second to kind of take a peek at the questions that we have there, and we'll circle back and take a look at the results in just a second. Until then, we are going to dig now deeper and zoom in on confidence check itself, because it itself actually has many layers to it. And at the top level, the 3 main categories are typo detection. So this one's pretty self-explanatory, but any obvious spelling or grammar issues are going to be caught by this layer. Next up, we have link analysis, which has several checks within it. So it's going to scan for broken or malformed links, so ones that just won't work at all. It's also going to look for broken anchor tags. Say if you're doing a menu within your email. It will look— it will check buttons and menu items for missing links. So that's a common one that can happen. You put down a button and you forget to add the link to it. It's also gonna look for placeholder links. And critically, and maybe most novelly, it's gonna look for links that it believes are wrong in the context of where it was included in the email. So what it's actually doing is it's looking at the URL itself, parsing through that, and looking for clues that might indicate that it doesn't actually match the context of the email where it's included. And this is common when you, say, paste in the wrong link or you paste the link in the wrong place in the email, right? This has definitely happened to, to me before, and certainly we've heard from customers that that this is an all too common issue. Last up, something that we've kind of, you know, are calling a consistency check. And this is just a general one where we're looking for any clues that seem off, right? Dates that may accidentally reference the past, a day of the week in a date that doesn't actually match the date itself. Numbers that just don't add up, logical errors. It's also looking for placeholder text, so some lorem ipsum or quick brown fox or to-do, and then any other consistencies really. So titles, names, products, brands that might be spelled or capitalized inconsistently throughout the email. So these are all the checks that we've tuned specifically for internal comms workflows where you have that duplicate and replace is really kind of a common workflow pattern, right? Okay. So why don't we maybe go ahead and see if our results are ready to go from our first poll? We can take a look at how things stacked up. So, all right, starting with question number 1. All right. It looks like it's a neck-and-neck tie for regular use for content in Q&A and policy-approved and org-wide. And that's, so that's great to see. And I think if we did the same poll, you know, 3 months ago or 6 months ago, we would have definitely seen a shift, you know, and probably the bulk being up more toward the top. And I think as we continue, as time passes, we're going to see that these, you know, first and second responses here, the values there will continue to kind of move into the lower brackets. So kind of a shift towards more adoption, right? All right, question number 2, how does your organization perceive the value of AI? All right, and so the lion's share seems to be here in the high strategic value and promising but cautious, and very understandable. And again, I think we'll continue to see that shift as AI adoption increases. we'll see that shift to having an even, you know, growing number in the high strategic value section. Which are the following— of the following are formally approved? So we see, okay, Microsoft Copilot being a standout here, ChatGPT Enterprise as well as non-enterprise. Interesting. And Gemini. Okay, is up there. All right, and trailing far behind is Anthropic's Claude. So not surprising there, but it's, it's really great to see so much AI adoption. And I think that really makes Confidence Check a natural fit for many of you, many of you on, on the call. All right, let me just take a quick look at the chat in case there's anything I'm missing here.

Speaker C: Nope.

Speaker B: We're looking good. All right. So now with that, I think that takes us into demo time. All right. So the exciting bit. Let's hop over here to Contact Monkey. So what we're looking at right now is our email editor. Literally what we're looking at is a mocked up version of a monthly newsletter. In this case, we're playing the role of Acme Global, a pharmaceutical manufacturing company. Not so important, but, you know, just to, just to say that what we're looking at is something that might be reminiscent of a monthly newsletter on the longer side, many sections. And we're about to hit send and we're going to run confidence check. And that is accessible up here in the toolbar in this case. So we have the lightning bolt button, pop that open. That is going to open up our confidence check popover. And in order to run those checks that we were talking about before, we just simply hit this button and off it goes running through those checks. It's doing multiple passes to check for consistency, link things up, looking for typos, checking through links and looking for inconsistencies. Typically this takes about 30 to 60 seconds. This is a longer email, may take a little bit longer, But as it says here, feel free to close the window. And why don't we, you know, in the meantime, while that's crunching away, let's go through maybe our typical manual proofreading process. I like to run through all the links myself, top to bottom. So I'll start with my first link here, pop that open. Looks like we're good to go. That's the financial reports, and that's what we have linked there. Scroll down to our next link. We have a news article here. We have another news article. So let's check that out. Yeah, looks good as expected. News article on regulatory trends and DNA integrity. And then let's, let's pop back here.

Speaker C: Alright.

Speaker B: And sure enough, we've seen— we see that the spinner has now changed to a green checkmark. That indicates the confidence check has finished its check, and we can now take a look at what it's found. We categorize— we group things together under these headings, so copy, dates and times, and links. So why don't we go ahead and just dive right into it? One thing to note, each result is presented this way with a subheading that gives us kind of a general classification, and then a 1-sentence description detailing why it's alerting you to this issue. So first off, it's found an inconsistency here in this button. So we're linking to an event that has November the 20th. But if we look at the URL that's included in this button, it has found that the time somewhere in this convoluted URL is actually November 12th, right? So already off the bat, there's an inconsistency and we have to look into this, right? I think in this case we might have gotten bumped because of the quarterly town hall. So we'll have to go back and take a look at the link to the calendar. All right, another example of an inconsistency here that it's picked up on. I'll give you a second to read that. Basically, the manufacturing reliability here is claimed to be 97.8%, but in our CEO message, he's talking about manufacturing reliability of 99%. So, you know, again, another thing to just kind of dig into and look into which number is correct. Next up is an interesting one. This one's kind of a personal favorite of mine. But in this case, it's— sorry, I kind of went too quickly there maybe. Let me show the message. The image alt text spells the name Ethan Peterson with an O, which is inconsistent with the spelling Ethan Peterson with an E here at the end, right? And sure enough, we see the E, and if we look at the alt text, it picked up on that clue that there's something that we should reconcile here.

Speaker C: Right.

Speaker B: Next up, you know, maybe an easy one, but the copyright year is listed as 2024. And naturally, that, that's out of date. What do we have next? So Avery Caston, it has picked up on the fact that she is joining. It says that she's joining the finance team. However, that conflicts with her listed title of manager of clinical operations. So obviously something there. I happen to know that, you know, this is the type of thing that happens when we took the description here for Ethan and moved it over to Avery. We made our updates, but we forgot to update the team, right? So that's kind of where that was inherited from. This is a good one that, again, kind of speaks to that core problem, one of those user stories that we go back to where an email was sent with an incorrect subject line. And so here, it's picked up on the fact that the email subject is talking about August, but the body content is November. And sure enough, if we jump over to our subject setting, we do have that mismatch.

Speaker C: Right?

Speaker B: Okay, I'll keep chugging down the list. What do we— what else do we have? Robert Rodriguez, he's our CEO. It's picked up on the fact that in one case we've spelled it without an accent, and in the other case we've spelled it with the accent over the I. So very subtle, easy-to-miss thing. But, you know, probably best if you, if you get the CEO's name right. A number of other typos. I won't go through each and every single one of these just in the interest of time, but suffice it to say that it's picked up on a whole bunch of other typos that we can simply click on. It will take us exactly there and we can update accordingly. Next up, under the dates and times, It's flagged a day-date mismatch. So that's what I was speaking to earlier. Here, November 20th is in fact a Thursday and not a Friday, as we kind of incorrectly referenced in the copy. All right, on to our last subsection here under links. So kind of starting with, you know, one of the more obvious ones, a broken link. So this will naturally be a problem. So when we pasted this in, we, we dropped the first letter, which is the H from HTTPS. So this link is gonna fail. Next up, in terms of some of the subcategories, we have that missing link. So sure enough, this button, as it says here, is completely missing a URL. So we'll go ahead and update that. The placeholder link. So in this case, our Learn More, we just popped in a to-do. So it's letting us know about that. And now we're gonna get into my favorites. So these are these— it's looked at the URL, it's parsed through it, and it's found cases where there's a mismatch that it's alerting us to. So first up, our Sunrise Yoga session, it has a parameter for a different event, which is the Lunch Pilates. So sure enough, we can check that out and confirm that yes, that is in fact the case. So maybe we just went down the list and just grabbed the wrong bullet, let's say, when we copy-pasted this event link into this button. And then it's gonna take us to those 2 news articles that we actually looked at, you know, these 2 here that are related to these buttons. And so it's actually pointing out there's something that we missed. And it's so subtle that, you know, genuinely looking at these 2 things, this is an example where it's very easy to miss the fact that these 2 are actually swapped. And that's exactly what it's telling us here. So the read more button for the top regulatory trends has a link to an article about DNA integrity, which appears to be swapped with another link in the email, which is the one below it. So this button here for DNA integrity actually links to an article about regulatory trends, right? So some great catches there by Confidence Check. Now, naturally, we don't expect that your emails would contain this many issues, but just to kind of showcase the full breadth of what it is tuned to catch. And it's just that additional confidence layer in your arsenal, right? So that's our demo. Now that we saw all the great things that Confidence Check can do today, just a call out for full transparency. to kind of set expectations, just some callouts on what it can't do yet, at least. So it's not processing any images or videos today. It's also not visiting links. So when we do the link validity checking today, we're looking at the URL itself, and we're not actually visiting the page and doing kind of a full scan of the content there. You know, naturally, there are some issues there with intranet links, Nothing that can't be solved for, but today we're finding a lot of good results just based on URL clues, right? It also doesn't detect inconsistencies across emails. It's limited to the scope of the email itself today. And lastly, intentionally, it doesn't provide any subjective feedback. So we've kind of— we've tuned this, this check to be specific to kind of concrete issues, all the things that we've looked at today, with potential for kind of looking at more subjective checks in the future. And just a final note, again, it is leveraging AI, and AI can make mistakes. So confidence check complements human review, but it doesn't replace your editorial judgment, right? Now, having said that, these are the 3 things that we have today, but there are many things that we— that are on the radar as additional checks that we can continue to build on top of our confidence check platform. And those are the things that we've kind of noted here off to the right in the gray. And we're gonna spin up a poll because we'd love to get some feedback and hear from you. What would you like to see, you know, next in confidence check? What's a priority for you? What would really move the needle? And as I do that, as we do that, I'll just kind of speak to each one of these just kind of briefly to give you a little bit more context on what they are. So accessibility check, naturally this is something that's scanning the email for WCAG-aligned issues. So missing alt text, insufficient contrast, heading semantics issues. Next up is tone analysis. So this gets into more of the subjective feedback. We can evaluate sentiment, formality, clarity, inclusivity, compare it to your prescribed, you know, tone of voice, and see if there are any mismatches there. Brand consistency would focus on the more cosmetic issues with the— with an email, looking at colors, fonts, spacings, and seeing if there are any violations with respect to what you've set out in your brand settings. Custom check— this would allow you as an admin in your org, let's say, to set up a custom rule. If you have certain phrases, terminology, rules that you have, you could specify one, and confidence check could enforce it for you. AI presend recipients— what we're looking at here or thinking about is AI personas that would mirror, you know, key audience segments of yours. as a way to get qualitative feedback on your draft before you send it out. Like, how would this group react or receive or understand your email? And lastly, deep link validation. So that's that kind of next level of link validation where we would actually visit the page, inspect it, and then give us greater, you know, clues on whether the content matches the email context. So I'll just leave that up and then speak to availability, and then we can circle back and take a look at the results. So we're currently in, in a closed beta, and we've begun rolling this out already to our first beta signups. Happy to say that everyone who has already signed up for the beta will be getting access at some point. We're going to be continuing to roll this out over the course of the next few weeks. Keep an eye out on your email. You'll be contacted via email with the details on how to activate the feature in the coming weeks. If you haven't signed up for beta and you're an existing customer and you're interested in doing so, it's not too late. We're going to pop up in the chat the link to the beta signup page so that you can go ahead and sign up. Otherwise, we're targeting general availability of the feature by mid-December. So just about a month away is when we're looking for GA on the feature. In terms of pricing, at GA, we will be giving all accounts 90 days of full access to what you saw today. We want to make sure that everyone has the ability to experience the value that Confidence Check can bring to your email drafts. And then from there going forward, it's going to be available in our Enterprise tier. And you can, you know, discuss with your customer success manager the details of what, what that means for you. And for future customers that are on the call, if you're interested in learning more, the follow-up email with the link to the webinar is going to include instructions on how you can go about doing that. All right, so why don't we go ahead and take a look at the results of the poll.

Speaker C: Quickly.

Speaker B: All right. Oh, brand consistency. Interesting. Okay, great. Great to see 70% on that one. Okay, so that resonated. That's great feedback. I guess next up would be tone analysis. Okay, great. And then after that, custom check. Okay, great. Yeah, so I mean, I guess resonating with the fact of, you know, we want to be consistent in our branding, the cosmetics, using the right colors, Spacing, fonts, having that consistency, definitely, you know, understood on that side of things. And next up, just generally where people are looking for help. So this is interesting, very even split across the board. So technically, what's our leader, I guess, by 1 percentage point? Understanding analytics. Okay, that's a great one. Yeah, definitely helping to kind of decipher all the numbers and understand, extract insights from your data, right? From the numbers. Okay, awesome. Thank you very much for all of that feedback. That's very, very helpful. So at this point, we're gonna hop over to questions. We should have at least a bit of time to answer, I guess, our top couple or few.

Speaker C: Yeah, sounds good. Thanks, Dan. That was a great overview. So excited for Confidence Check, and thank you everyone for your questions. I'm gonna start. I've seen this one come through a few times. A couple people asking if it looks for extra spaces.

Speaker B: Oh, okay. That's a great one. I think that's not something that we've tuned it for right now, but that's a great bit of feedback that would be an incremental enhancement on top of our kind of existing editorial rule check. So yeah, generally speaking, we have tuned it to kind of have a pretty high bar to make sure that we're not getting a lot of false positives. So anything that, that might be, you know, egregious gaps, it will likely pick up on, but maybe some trailing whitespace and whatnot, it might not as it sits today.

Speaker C: Perfect. Thanks, Dan. And here around if the tool will be able to identify ADA compliance requirements. And just on that note, a couple of questions around if we have any technical documentation related to AI that we can share?

Speaker B: Yes, I'll start with the second part of that question. It maybe cut out a little bit towards the beginning, so maybe we'll circle back to the first part. But regarding technical documentation, we do have our AI policy published publicly on the contactmonkey.com site. So we are trying to be very transparent about our usage of AI, which features use AI, which subprocessors we're using for that. And I will underscore the fact that any of those subprocessors that we're using, we are never training any models based on any of your data, you know, period, right? Furthermore, we are only using subprocessors that have a zero data retention policy, so none of your data is being retained by those subprocessors. And also with Confidence Check, it is an opt-in model. So it will ship as off, and then you will have to go to the settings, and an admin will have to explicitly turn it on in order to enable the feature in the org for all users. It can be turned off at any time. And on top of that, I would just call out that Confidence Check does not, and does not have the capability to modify your email in any way today.

Speaker C: Right?

Speaker B: So there, there shouldn't be any concerns there that maybe, you know, Confidence Check is, is making a modification to your copy and it's incorrect because it's only alerting you to issues and using your editorial judgment to make corrections.

Speaker C: Thank you, Dan. For the beta, is this user-specific or would this be across accounts?

Speaker B: This would be, if it's, if you're participating in the beta, it would be across your org, your organization. It will be turned on for your organization. So all users in the organization would have access to it. And it is applicable and open for signup for all of our existing customers' accounts.

Speaker C: Perfect. I think we have time for one more. Yeah, it'll check review for style, specifically AP or Chicago.

Speaker B: I heard— sorry, I was cutting out there again a little bit, Rebecca, at least for me. Do you mind repeating?

Speaker C: Yeah, will it review for Chicago or AP for style?

Speaker B: Okay, yeah, that's a great question. It isn't tuned specifically for AP or Chicago for style. Again, this is great feedback though, and it might manifest itself as potentially a separate check. Again, we just had it kind of tuned to any obvious typos and grammar issues, but that's something that we could definitely, you know, consider and look into for, you know, a future iteration.

Speaker C: Amazing.

Speaker B: Yeah, yeah, it's great feedback.

Speaker C: Awesome. So for any other questions, hopefully everyone can hear me okay. We'll follow up with you via email and feel free to reach out to your customer success manager if you are an existing customer and we can direct you in the right place. But I will pass it back to you, Dan, and we can wrap up.

Speaker B: Yes, I just wanted to say a big thank you to everyone for joining today. For those of you that signed up for the beta, Really appreciate your partnership in helping to shape this feature. It's very exciting to see it go live and already catch real customer issues and drafts. You know, it's great getting those reviews from customers saying, oh my God, it caught this issue that would have otherwise snuck out the door. So again, for those who've signed up for the beta, keep your eye out on your email for an invite over the next few weeks. And thank you very much.