Revolutionizing Customer Success with AI
Philip Kibby of Moveworks showcases how AI capabilities simplify employee interactions, offering immediate time-savings and insights that drive productivity and improve customer outcomes.
[Music]
As you promised me that I was more than all the miles combined
You must have had yourself a change of heart that had played through the drive
Because your voice turned off exactly as you passed my exit sign
Kept on driving straight and left our future to the right
Now I am stuck between my anger in the plane that I can't face
And memories of something even smoking weed is not in place
And I am terrified of weather 'cause I see what it brings
Talk told me to travel but there's COVID on the planes
And I hover more and put it to see even other sticks
And I saw your mom's chief forgot that I existed and it's half my fault
But I just like to play the victim I'll drink
And I'll go home and tell my friends come home for Christmas
And I'm dreaming of some hurting you that I might not have
But I did not lose now your tired tracks in one pair of shoes
And I'm splitting half that I don't have to do
[Music]
So I thought that if I piled something good and all my bad
Can't stop the darkness I inherited from that don't I am
No longer funny 'cause I missed the way you left
Once called me forever now you still can't call me back
And I hover more and put it to see even other sticks
And I saw your mom's chief forgot that I existed and it's half my fault
But I just like to play the victim I'll drink
And I'll go home and tell my friends come home for Christmas
And I'm dreaming of some hurting you that I might not have
But I did not lose now your tired tracks in one pair of shoes
And I'm splitting half that I don't have to do
Oh that I don't have to do
My other half was you
I hope this pain's just passing through
But I've doubted the time of her mom
But it's just to see even other sticks and I saw your mom's chief forgot that I
existed and it's half my fault
But I just like to play the victim I'll drink
Good morning everybody I think we have a bunch of attendees here today
Phillip if you could go ahead and share your screen for me
Welcome everybody to our CS customer spotlight
Today we are going to talk about scaling CS
Using AI and fill up the next slide if you could
Using AI and fill up the next slide if you could
Using AI and automation and our special guest today is Phillip Kibbe who is a
customer of mine at MoveWorks
And Phillip as you're going to see in the next slide
Has had an interesting career that brought him to this point
So about 13 years ago he was working for the US Senate Ethics Committee
And he was going to be a French wine importer and distributor
And now he's working with MoveWorks as their director of CS operations
And when he gets up in the morning he's super excited about streamlining
everything for his CSM
So that they have a lot of automation in our lives
So they can focus on the higher value conversations that they have with
customers
So with that I'm going to turn it over to Phillip
Keep those comments coming in the webinar chat
And one more thing for Q&A
Let's use the Q&A button on the Zoom meeting
And I will keep an eye on those as well
And if it's something that is pertinent to the conversation, Phillip will stop
and answer
If not we'll hold those to the end
So Phillip I'm going to turn it over to you and thank you everybody for joining
Awesome, thank you Denver
Yeah, so again welcome everybody I'm Phillip Kibbe as in
We're set I've been at MoveWorks for about three years now
Starting as a CSM
And it's been pretty amazing ride so far
The team has grown from 15 or 5
Yeah 15 people total in the CS to 75 in the three years I've been here
So it's been pretty crazy
And as Denver reference I have a pretty odd career
This is my third career within a small one
Started off in the US and ethics committee advising
Senators on public financial disclosure laws and rules
And then my wife and I put our jobs and moved to Paris for a year to study
I studied wine, she studied pastry
And then I started a wine import company before turning back to do some
technology work
And it's been a pretty cool ride
And what I like is able to see all these different aspects of
You know government and then you know hospitality in wine importing
Which is huge implications for an application to customer success
So we'll see someone that kind of weave through how we think about CS
So moving on
I wanted to go through today's agenda real quick
So start off just introducing into what moveWorks is what we do if you've never
heard of us before And we'll talk you through our gain site journey
And then going into how that's layered into our own automation
And internal efficiency initiatives
And we'll kind of highlight four of them that we've done the last eight months
That have quite a bit of impact
And then I wanted to end my steps you can take today
To really go and drive impact in your organizations
And if any of these things resonate with you at all
So moveWorks
So moveWorks at its core is an innovative AI platform
That unifies all enterprise systems
We are huge technology leaders in the space
We've been using large language models and GPT before
They were cool for years now
And as you know evidence by the fact we're in the cloud 100
We've been I think four years running on AI
For the AI 50 list
Winning things like AI breakthrough awards
And not just the technology side
The company itself has won quite a few awards
For just being a great place to work
But on Newsweek's Best Places to Work List
For a couple of years now
As well as Forbes' Best Startup Employers List
So it's been a great place to work
And as you'll see given lots of freedom and resources
To really drive lots of impact
And the organization
And what we do as a product
Is we offer a conversational AI
And generally AI platform
That unifies employee services
To a single chat interface
So employees can access the information
And query information form actions
And really get the help that they need instantly
Versus having to go through
Kind of all the legacy ways
To get help in an organization
Where leaders in the organization
We offer lots of advanced analytics
So you can understand
What your employee experiences like
Where you can lean in and make things more
Fishing and better for your team
Some of which we used human and
Identifying areas of opportunity
In our own automation efforts
And lastly, developer friendly workspaces
To extend the product, right?
We have offered lots of native integrations
But we really want to make it to where
You can build your bot for
And now co-pilot, right?
For what your organization really needs
To help improve employee service
And just real quick what that looks like
Is functionally on the left
All sorts of questions you may have
About the help you need to get an organization
From what's our 401k match
What's the status of?
This headset I requested
How do I reset my multi-factor?
We take that
Those natural language queries
Run it through a whole host of things
Like different machine learning models
Or own data
Using things like your role in
And then query lots of native integrations
That we have into your system
Before returning to an answer
That solves your problem
And it does all that
Pretty instantly
So it's a really cool platform
And obviously now has more and more implications
With the kind of the rise of copouts and GPT
So that's a lot of what we do
And you'll see the fact that we're so focused on
Employee service and making sure that people
Get the help they need quickly
That really just is infused in all the things
That we're going to be able to do
And we're going to be able to do
That with all that, we're going to be able to
Get a lot of information
And we're going to be able to do that
With all that said
Why do we end up using GAY
Inside and using it as a platform
That we want to integrate
With our own platform to drive
Efficiency and CS and automation
And really when I started
At Moverz, we had a team of about
Seven CSM.
And we had a team of the CSA team
Who was doing support and
Random operational things, so very small team
But by the end of 2021
That team had grown from seven CSM's to 19 CSM's
Right around that time I started the CSOBS team
Because we definitely needed operational support
And you could start to feel the pain of a lack of a centralized system
And ways to access information
And then six months later by mid 2022
We had 25 CSM's
And we needed to procure a centralized system ASAP
It was just really critical now
That information was all over the place
In the organization
And as you can see here's a slide
We actually used in our pitch to leadership
To get known, not only by a CS platform
But to recommend GAY
Inside, which was like CS tools and platform
And a process was all over the place
Notes were being taken in five different surfaces
Account info all over the place
Customer issues were recorded in
Zen desk, but they're also stuck in notion
And Jira, not even Slack
And so we didn't really have a way to
Unify all this information
Into one system that allowed CS to see
The state of their accounts and take action on those accounts
At scale, and that really necessitated the need for a platform
And as we look through all the options in the market
We landed on GAY insight for what I think is about four primary reasons
The first of which is there's a robust
You know the most robust data model we saw of all the CS platforms
And this was a huge differentiator for us
We deal with lots of data, we work on lots of data
In a very data driven CS organization
And so we needed a system that
Would allow us to, you know,
In just the data that we used to
Day with how to do a bunch of new transformation
Before it entered a platform
And then once it's in a platform, transform it
And combine with other data sets
To create unique data for us to understand our accounts
The second was total flexibility in configuring the product
So this is also super important
And we needed to conform the product to how our CSMs worked
And manage their portfolios
Not conform how CSMs worked to the software platform we purchased
Absolutely critical that we met CS whether or not
And then asked them to change how they worked
We're very successful at the CS team
Have really high GDR and we didn't want to mess with that process
So we needed a platform that had lots of flexibility
And with that comes integrations
As you saw on the slide before
There's actually quite a bit of
Systems at how's information about customers
That we needed to unify into one place
And so having a robust native integration set
Was really important
And particularly native integration
And things like snowflake
Was kind of a no brainer for us
That allowed us to move all this data
About the product
And all of our go-to-market data in snowflake
To really give us a very easy way
To get this information in front of CS
And then lastly was just more robust account resources
So throughout the whole sales process
The team was really responsive
And it was very clear that
Gainsaw would be great partners with us
And even now that we are
Customer is you know, a slang dimmer for the webinar
Apologies for how many requests we send
His way and Ashish
As our technical partner and both of them are amazing
And always willing to blend a hand
And so that since the partnership
Was really really important
And so these are the reasons we chose Gainsaw
And you can see as I go through the rest
Of the presentation
How we layer all this stuff into Gainsaw
Not everything is housed in it
But it's used as a conduit to make things
Easy to find and reference as well
So this is really why we chose Gainsaw
Which really helped us
Kind of launch this
Motion of efficiency and automation
We purchased it in March or May of 2022
And we went live with implementation in September
And one thing I'll call out of
We launched with a very like V1
Not very robust systems
We could build it
Based on feedback that we got from CS
And that was really the right call for us
As we really ended up building something
That CS really wanted to use
Versus spending 8, 12 months on implementation
And not actually landing it where we needed it to be
So from here
What I want to talk about is our
Automation and efficiency journey
So as was referenced in
The sign up materials
We ended up saving over 4000
hours in the last 8 months
And wanted to walk it through
How we did it and I wanted to focus on the
Fore highest impactful initiatives that we did
On an automation and efficiency front
In order to deliver against these goals
Particularly as I'm sure everyone is aware in the last
You know, years so the map environment for SaaS
Has been difficult
And a lot of startups had to shift from
You know, hyper growth, grow at all cost models
To something that was more efficient internally
To start containing costs
And that was really something we wanted to do as well
We'd grown the CS team exponentially
You know, in the span of
12 to 18 months
And now we actually needed to make sure
That those people we hired
Had all the tools and that they needed to
Really manage accounts at scale
And so this was really our focus
And the last, you know, 8 to 12 months
And it's an 8 month period I'll focus on here
And all the things that we were able to deliver
To meet those goals
And from an impact perspective
Over an 8 month period we, you know
As I mentioned, saved over 4000 total CSM hours
And we did that by
Automating nearly 6000 calls
Nearly 3000 emails
I'll talk about this deck generation
That's automated deck generation that we developed
And we've seen almost 1500
Over 1400 decks automatically generated
In the span of 8 months
Which resulted in a 3x improvement
And joint success plans being developed
For our customers, which was a massive win
And our focus on customer success
Or customer education
Actually a 10x reduction in CSM time spent
Directly educating customers
And this resulted in an 8 month time frame
Of CSM's increasing the AR
They managed by 32%
Which was just a super big win for us
Because it, you know, made us realize
That all these efficiency things allowed us to really drive
Scale for our team
And allow our CSM's to take on a bit more
But do so in a way that wasn't overwhelming them
And also generated and delivered customer outcomes
That we were looking for
And so I want to kind of talk about
How we generated this impact
And, you know, first I'll start with
Call of Summarization, which was one of the first things
That we did
And the problem we were trying to solve
Was, call notes were being stored
In all sorts of locations across
The, you know, our tools, right?
Some were in Apple Notes, some were in Google Drive, some were in Notion
And it was just becoming a mess. It's hard to reference
If someone was out of office
You didn't know where their notes were stored
If the customer question that came up
So we wanted to create, you know, that was a big problem we were looking to
solve The second was just clear and consistent
Call of Summarization
And this is clear and consistent
Call of Summarization
Consuming to create, you know, as a CSM
For many years and you're on a call
You're trying to listen to your customer
You're jotting down notes and they're often all in shorthand
And so in order to, you know, create good notes
You have to actually spend time in the day
Just cleaning them up and making them readable
For a third party
You know, third was, as a result of this
We seeing action items would get missed
And we were talking about situations
But missing some action items
Because you're a so engaging conversation
Resulted in an downstream customer impact
Or the customer was like, hey, I thought we mentioned this
But, you know, CS didn't quite grab it
And so CSMs were often left with a choice
Right, optimized customer calls for better customer interactions
Or optimizer calls for taking better notes
And action items and really the
For us the answer was obvious
We really want them to optimize for better customer interactions
So what we did for a solution is we wanted to create a single repository
Which is gain site
For all call notes for customers
Make it really easy for us to store them, search them
And understand what's going on at an account
The second is we wanted to leverage
In an AI and automation to summarize those
Call notes in a consistent format
Right, this is a big time-sater for us
And we wanted to overfit the language model
For robustness and action items
So that nothing would get missed
And so the solution we created started back in January right after
The release of a GPT 3.5
Otherwise known as Chat TPT
And something that we were able to deliver
In about four to six weeks
And on the left you can actually see
What this looks like in gain site
Of course I had to blur out the actual call information
This is all custom, this is your real call summary
But functionally you see that
And we store in Salesforce the call summary
We push it to gain site
It has a summary of the call that's automatically generated
A whole list of action items that are automatically generated
And then we link out to the gong call
And the Salesforce record
And the way we did this was
We realized we had all the gong audio files
And gong, and we work with our go-to-market team
On getting access to them
And then we worked with our machine learning engineering
And product management team to say
Hey, how can we design a solution here
That was in the end automated for CS
So we write a script that grabs the audio files
We use OpenAI's whisper
Large language model to transcribe the audio into
Text transcript
And then we
Supplement it with some of our own internal data
Making sure that the model
Before we go to summarize the models understanding
Things like acronyms and customer names
And because our product is typically white-labeled
Our customers, it's not called Mover
Set a customer, they design their own bot avatar
And their own name for their product
Those bots have lots of personality
We wanted to make sure that the model understood
Those names so that they didn't misinterpret it
And then we summarize using GPT 3.5
And then we push that summary of that transcript
Into Salesforce using work-auto
Which was in a partnership with our go-to-market ops
And Biz systems team
And then we use the native Salesforce to gain site integration
To push that call-noted summary to gain site
All this is done
You know, generally within about four hours of the call ending
We can always speed it up, but just from an engineering capacity perspective
And we wanted to keep it about four hours
But this is a hugely popular feature that we develop
You know, getting over 6,000 calls
Of an automatically summarized
And probably between when I pulled this data and now
We're doing about over 100 calls a day
And it just really took a load off of CS because now
Some, mostly, I still take their own notes
But I think that's a great thing to do
Because now, some, mostly, I still take their own notes
But now they can actually supplement them with something summarized for them
They can use as action items or write
Follow up emails and customers and do that very quickly
So it's just a huge time saver
And just something where it was a piece of mind of knowing that your calls
would be summarized There would be a history of them when we do handoffs
This is really huge
Because you can really understand what's happened in previous calls
And then we push summaries of quarterly business reviews into Slack
Automatically so that anyone who's following the account can see what happened
in the last QBR call
So this was a huge win and produced quite a bit of time saving on its own
The next initiative we wanted to focus on was automatically sending emails
And again, the problem we were trying to solve
Was that programmatic emails were still being sent manually by CSMs
Using email templates we created in Google
And then sending them using those templates and still sending them manually
Because we wanted to keep the emails personalized
And even with email templates, this process is pretty time consuming
If you even, you're copying from a template, you still want to double check
everything
It's probably three to five minutes in email
And if you're doing this at scale, this just ends up taking quite a bit of time
And it's still prone to error, right? You may copy the wrong template, you may
forget the swap out of customer name
And then, you know, it's kind of a mess
And also at its core, right?
Sending programmatic emails is just low leverage, right?
It's not really something that's going to provide CS a lot of leverage
So we wanted to solve for that
And then also this last, at the beginning of this year, we spun up a new
commercial CSM team
To operate more at scale
And so that necessitated the need for more automated communications
So what was our solution here?
Well, we turned to GainSight for this
Which was we leverage GainSight's journey orchestrator and email assist
functionality
To send programmatic emails at scale
And this allowed us to build more sophisticated email programs
And particularly to help the commercial team
I want to say it's sophisticated, they're still quite simple as you'll see in a
second
But it allowed us to kind of chain together multi-week programs, which is
really important
So how this looks in practice is, you know, journey orchestrating
Actually see an example of one of our journeys at the top of the page here
And what we've learned is that simple workflows work best, right?
Nothing too complicated here
This is an example of a three-week kind of new customer after launch
Campaign that gets sent on the commercial team
We use query builder and contact attributes to create automatically recurring
audiences and campaigns
So these are often set it and forget it
So there's a lot of scale and leverage you can do by spending an hour or two
Setting this stuff up and then it just kind of run in the background
And some examples of campaigns that we send, we do welcome emails to services
like our academy and community
Customer onboarding as well as we call cohort based campaigns
So if the customer falls in a certain cohort, whether it's industry or product
or product performance
We can then automatically spin up a campaign for them and just get stuff sent
to them
Which is really great and something that we're scaling to all segments, not
just commercial
And the second thing was email assist
You actually see a real example here on the screen of an email assist we do for
NPS survey responses
So we send an NPS surveys and twice a year to our customers
And it's imperative that our CSMs follow up on every customer survey response
So that, you know, ultimately if you don't follow up and acknowledge and think
the customer for giving feedback
They're going to stop giving it so
But when this is a manual process, it's hard to get to be this at scale
And so we use email assist to send a CTA once a survey response comes in
And CSM will just pull up this, it's tokenized with their customer name
And then they can just insert any relevant comments as necessary and then
quickly send off the thank you and response
And that really increased CSM survey follow up from around 60% over 90% which
was a huge one for us
And then we also use email assist for promoting marketing events like webinars
like this or in-person events
As well to give a personal touch and not simply rely on sort of mass marketing
emails to do that
So these have saved quite a bit of time actually, it's kind of crazy to see 3,
000 get spun up over an 8 month time period
But, you know, again once you set them up and they run, it really just runs at
scale
And it's just a huge benefit to the team and the CSMs love it
It actually ends up engaging customers who don't, maybe they don't want to show
up on calls every week or every two weeks
But they will respond to these automated emails and that's been a big win for
us
One of the biggest things we did was deck automation
And this is something that I'll spend a bit of time talking about because this
was probably had the highest impact
It was from a time savings perspective, but I think quite a bit of customer
impact as well
Which the problem we're trying to solve here is taking CSMs on average 90
minutes to create a joint success plan
Or quarterly impact, which is our version of quarterly business review deck
respectively
And this really varied based on your 10 year and familiarity with our data
And if you were new to move works, one of these decks could take you 3 hours
It was a huge time suck
And that meant the decks were very inconsistent from CSM to CSM
So quality was quite hard to control
And it meant that customers were getting really inconsistent, you know, QIRs or
JSPs
And the success plans are the foundation of our CS model
We have to, you know, lower success plans to our customers
And they have to be really high fidelity
And live with this kind of really inconsistent quality
Building them was also compounded by internal dashboard or slow
If lots of CSMs were hitting the same data table to get data
Of course that table is going to operate a bit more slowly
And then there is no programmatic ways to insert slides promoting events or
product updates or customer stories into decks
Which meant that a lot of our decks were just missing critical things
Because CSMs, you know, didn't remember that it was stored in some random
marketing folder
And so, and you couldn't blame them, there was lots of places to restore it
And it was just hard to find
And so this allowed us to make sure that every deck had a certain, you know,
level of quality and announcements in it
And ultimately we were spending too much time building decks and not enough
time preparing
For these meetings with customers which meant pre-flighting all this
information with the customer
Working with them just to refine these plans and presentations to have high
impact
And so, you know, ultimately we needed to solve for this
And our solution that we built, which I'll talk about a little bit in more in
depth in a second
Is a fully built automated deck in less than 60 seconds
We wanted to just make this so seamless, we were able to deliver on this, which
was really amazing
We wanted to standardize our JSPs and QIRs, but also still leave room for CSMs
or customise them as they see fit
We really don't want you to take the automated version off the shelf and
deliver it
We'll deliver you something 80 to 90% complete
And then rely on you to kind of put that polish on it to make it really
customer-specific
We wanted to standardise, we were able to standardise on data and metrics
So that the standard story and narrative in these decks was quite strong
It didn't rely on you being an amazing storyteller naturally to deliver a great
presentation
And of course, increased the quality of visuals and the baseline storytelling
Which then freed up CSM time to spend preparing and pre-flating this
information with customers
And so, ultimately the solution offered us a significant improvement in
information and data that we could provide to our customers
And so, what this looks like in practice is actually quite complex and was a
huge cross-functional initiative
But we have a data science engineering team that built a tool called Lego Land
internally The idea was that each piece of data is a Lego block
If we could build a modular system of building Lego blocks any way we wanted to
We had a really powerful tool to operate and provide data reporting and scale
We then once the team built that solution
And I'll talk a little bit about how it came about and how we even figured out
there building something like this
We then use super blocks, which is like a no or low-code app builder
Functionally that we use a lot internally
And that was a partnership with data science, engineering, product management,
customer success And the screenshot you see is an actual example of what a CSM uses
They come into the super blocks and they select the org they want to generate
the report for
We automate a handful of different reports now, not just these two
And you select the report you want and click generate report
And there's like a time period that will come up
You'll select how long do you want the data to be analyzed for on the deck
And then it will generate the report for you
And the benefit is that everything is customized to your account from the
customer name to their bot avatar
All the data and charts and graphs are custom to what is at your customer
Even the configuration of the product changes what we recommend to the customer
to Do to improve performance of their product
So everything becomes this quite automated, elegant solution
And what ends up happening when you click generate report are two things
One is you get the fully built deck in Google Slides
And that you can then use to edit and refine
And then we also send a link to that generated deck into GainSight
And this last piece was really important as well because
Especially leaders in the organization were always looking for decks
They're like where is this QIR deck? Where is this JSP deck?
I wanted to take a look at this next one
If there was a customer escalation or a question that came up during an
executive meeting
With that account, you just wanted to make it really easy for a leadership team
to find information
And so you didn't have to go, you know, you have the whole history of decks
that are generated
Inside a GainSight that you can just click on and get instant access to the
deck
And this was a huge way and not only just from a time saving perspective of
generating decks
But also from a perspective of just finding information internally and being
able to reference them
Or even our own preparation that we were doing for accounts
So, you know, I'll talk in a little bit too
A little bit kind of how it's all came about
But this was a huge, huge win and I think there's tools you can buy, a SaaS
platform you can buy off the shelf
That do this deck generation as well
So if you don't have a team internally to build it for you, there's other
options
But I think this was a super high leverage thing we did
That's provided just a massive amount of impact
And also learned quite a bit on how to talk to customers about data and present
data to customers
Now that we have something standard and that we can now measure against it
So this has actually allowed us to build really great refinements to these
decks as well
And the last thing I'll talk about from an initiative perspective that we did
before getting into some tips on what you can do today
Is customer education, although not technically like automation per se
I think this is a digital CS motion that has just yielded massive results for
us
So the problem that we are having is depending on CSM knowledge and tenure
Customers were receiving inconsistent education about our product
This is natural, if a CSM has been here for two months, you're just going to
know less than someone who's been here for two years
And CSM was spending on average over five hours, like five and a half hours a
week educating customers on our product
Which was a huge time suck and we didn't realize we were spending this much
time until we did a time tracking initiative
And we're really blown away at how much time is being spent here
And there was no scaled way to teach customers about the product or best
practices
So what do we do to solve for this?
A solution was to build a dedicated customer education function, which was a
team of one until about two months ago
And which just became a team of two
And we wanted to build a multifaceted education approach that had a dedicated
academy
webinars, office hours and things that we'll talk about
In a second, and then we launched a live cohort based training program for new
customers
Which we shamelessly stole from Tim Van Luu at Beansite
With the accelerator program and Tim was very nice to even advise us on his
learnings Which allowed us to build a really successful program
And this will call moverx fundamentals
Bottom
And this cohort training program, we've done not four cohorts
And the attendance has grown 3x over time
So this has been a really strong offering for us
We get a lot of great feedback from our customers on this program
We also have a dedicated academy, which is the screenshot
You see as an actual screenshot of the academy as it looks like today
Which is important for self-paced learning
We have dedicated courses by persona, product, we do Quick Start
Which is kind of an automation that we've just introduced
Which is you can go from learning about the product to actually doing something
in it In about 5 to 10 minutes using these Quick Start courses
Which is really great
And we also host a series of product webinars in partnership with our PM team
And those are done on a weekly basis
And the benefit is that CSM time spent
Went from over 5 hours a week to half an hour a week
In the 6 months since launching academy
Which was just massive in terms of time savings
And just really up level the quality of education
Our customers are receiving our product
So CS education, although not traditionally an automation
Has done a huge for us to scale the CS team
And of course we send information about who's signed up
And who's using it into gain site using an integration there
And then we also just launched a community using the gain site
Inside a digital hub product
Which has some interplay with academy as well
So lots of ways we're trying to create automations
And efficiencies between these surfaces
So these are the four main initiatives that provided impact
And again I just wanted to throw the impact on the screen
And outside of just these numerical things and volume
The actual impact back to business was pretty big
All these things we calculated from a soft time savings perspective
In terms of putting a dollar value to an hour that CSM works
This is about $750,000 in soft savings back to the business
But more importantly it's reduced our customer retention cost
Over the over the eight months
And that customer retention cost reduction has given about
Two and a half million dollars back to the business
Meaning it's two and a half million dollars cheaper to retain our customers
Than it was eight months ago for the business
And then from there, although we've definitely improved
Automation and efficiency internally and given money back to the business
Which was pretty important from our operations perspective
We've also really improved customer outcomes in terms of now
We were more communicative to customers
Success plans are much stronger
Much clearer signal on what to do and how to do it
And so ultimately this has been a really successful initiative for us
What I wanted to end on before opening up for Q&A
Our steps you can take now
Obviously if anything is resonated with you
You may not have all the resources we were lucky to have been
A company and we have ML engineering on staff to help out
But there are things you can do today that even can drive impact
Before you get into things like building automated deck buildings and things
like that And so I wanted to leave you with six things that you can do
And the first three are get cross-functional and do it now
I think you absolutely cannot do this alone within CSOps or just customer
success You have to be cross-functional here to succeed
If you're in operations day or you're in customer success and you own these
initiatives
Just set regular check-ins with your cross-functional peers
And product and go to market ops, data science, marketing sales, etc
And just understand how you can solve business problems together
You can't really scale your impact unless you bring the whole company with you
And what's really important is also aligns the company towards customer
outcomes
Which is really, really critical anyways
And so this is actually how we found out our data science team was building a
deck builder
Is working with our product team, we're trying to figure out what to do
To solve some of these problems and we had a conversation with our data science
team
They mentioned they were building something for this very specific sales use
case
And we started chatting with them and next thing you know, they're building our
joint success plan on an automated basis
And developing that product for us, which was just huge
And we wouldn't have known that if we weren't talking consistently
So I spend almost half my time just cross-functionally now talking with other
people
And just making sure that everyone is aligned on the same outcomes we're
looking for
And you do that by figuring out customer pain and customer success pain
You know, speak with your CSMs and CSM leadership to understand pain points
There for sure will be open to telling you about it
And if you're in operations and you don't have regular one-on-one with CSMs
You should start them now
My whole regular one-on-one with a handful of our CSMs and not only, you know,
is it just great to chat with them
But I understand their pain really quickly and I can also probe kind of
We know what the pain is with customers and how we can figure out how to make
improvements to the process internally
And to our customers and it's really helped create this amazing feedback loop
That allowed us to improve our offerings and services much faster than if we
weren't having these conversations
And a big thing we did was track CSM time
This was quite controversial when we did it, no one likes tracking time
Especially if you're not in some kind of like fee-for-service model, which we
are not But we were able to kind of get the buy-in from leadership and the CSM team to
do it for four weeks
This gave us such an amazing data set of where CSMs were spending time
And that allowed us to translate that pain into business impact, which is the
third thing here
In order to secure resources internally, you must articulate the business
impact to leadership across your entire company
This is absolutely vital, and so what we did is we used CSM tracking and time
savings projections to bolster a quantitative argument for resourcing
And then we combined that quantitative argument with more qualitative
storytelling about how the pain that we were experiencing internally
Based on our processes not being efficient, we're actually impacting customers
And those two things together make a very compelling argument to act
And it really allowed us to secure resources, and this one time tracking
initiative, although quite painful for everyone involved having to do it
We were able to use this data for six plus months to continue securing res
ourcing to improve customer success sufficiency and automation
And the last thing I'll end with is just start with one conversation, all of
these things outside of education, which was separate
Just started with a single conversation with a PM at Mover, so one of my really
close colleagues, Andrew and I started having conversations back in August 2022
about CS sufficiency
And at the time we were talking things like automated call notes, automated
decks, sending thousands of personalized emails automatically or all of Patreon
, they were not in our frame of mind at all
And our original plan was just to build an easier to read dashboard for CS to
leverage to create these reports
And just the more you start having conversations, you just start inertia takes
over and one conversation begets another, begets another, and if you especially
if you expand that to cross functional conversations
You really end up getting a full vision of what you can do in your organization
and how you can impact both CS and customers in a positive way
So although this seems like a lot of stuff and it was quite a bit of work and a
huge cross functional initiative, it all started just with a single
conversation trying to ask questions and figure out how it can make an impact
And so you have to start somewhere and so just encourage you if this is
interesting or resonates with you at all, just figure out a good partner in the
organization it can be and even in CS and say, hey, how do we start this today
and start ideating
and figuring out what you can do? And what was important is that we prioritize
for quick wins and then use those quick wins to give us justification and
energy to build on that progress
So instead of trying to automate all the decks that we were doing, we just
focused on one at first, which was a joint success plan and we poured all of
our resources into doing that really well
Not only did we learn a ton by just focusing on that one deck and getting it
done really well, because the initial deliverable was such high quality and
resonated really well with CS, it made everyone really excited to do more and
made us really motivated to do more as well
And we could take those learnings and now we work much faster and automate
decks we probably have five or six different decks now that we automate and but
it all started with just focusing on one and prioritizing in that done as fast
as possible
So as you saw, most of our automated email campaigns are very simple workflows
and they can be spun up very easily. You know, those email assists can probably
be done today if you wanted to spend up one to promote something to your
customers
So we start simple and just provide lots of quick wins and, you know, wash
rinse repeats. And then we also focused on other quick wins and I really
focused on four really big impactful initiatives but we do lots of other small
ways to be a life improvement that people notice and they really love and that
's allowed us to continue doing more because we're just not always focusing on
these humongous things that take weeks or months to get done but we're also
within those timeframes
Shipping these small wins. So for instance, we integrate gain site within our
own product and chat interface so you can chat with our bot internally and ask
who's a CSM of this account or what's the top ten ARR customers
And when, you know, what are the all the renewal dates in the next month. And
that was just a big win. These aren't data points that take people forever to
find. You can go into sales course, get inside it and get, and get to them
within a few minutes but it's just so much easier to chat with it in, you know,
with our chat product. You can take this information and transform it and
And then we also do lots of automation and we use our Slack as little and
really every customer has their own dedicated channel the whole team in it. And
what we do is we use a tool called rattle to send all these notifications. So
you get reminders on, you know, hey, two weeks until your next QIR is coming up
or, you know, we send, you know, we have a big event on the website.
And so you have a big event on November and every person that registers from
your account, you get a notification on who registered, you know, they're like
a virtual person, etc. So all these things, you don't save like a ton of time
on their own.
But they're all not really a great quality of life wins and that just gives us
lots of, you know, juice and collateral to keep doing more.
So read back impact to your organization. You know, you must develop ways to
track impact of anything that you do and regularly update leadership on the
impact of those initiatives that you're working on.
And, you know, we do that using a lot of reports and gains I mean we take a lot
of data from all these systems or, you know, we're able to track the impact and
then I take those reports, download them and we do some, some custom
calculations and Google sheets to build this impact reporting and we make sure
that it ends up everywhere.
And it's not to like tutor on horn or say hey we're so amazing but it's to show
the organization that the investments that they've given and the faith that
they've given in our team to to go and drive impact is actually working.
It's not easy getting a machine learning engineer to help at a machine learning
AI company in the most sort of frenetic and sort of innovative time and AI
right but we're able to show the impact of having an engineer help us.
And so, you know, the science team there's lots of data science needs across
the whole organization not just in CS but our ability to articulate the impact
and say the resources that have been dedicated this have really been impactful
and really delivered on on results.
Just help so, you know, impact reporting makes us way into reports that we do
for leadership whether it's executive leadership or own sales leadership.
And channels dedicated to these initiatives with regular reporting. It's almost
impossible not to know how these things are doing and that's just our way of
saying look, when we're very appreciative for the resources that were given to
us, especially cross functionally.
And we've used those resources and really delivered on impact and that gives us
the ability, you know, other leaders in the company go oh yeah we definitely
don't mind giving you more resources because we know it's going to be used to
drive impact and also teams like working with us because
we know that we're going to share out the amazing results of the stuff that
they do.
So really encourage you to develop these kind of tracking mechanisms and figure
out ways to update your leadership.
So that is the presentation I think right on time here so I see some questions
have come through so Denver do you want me to go through one one you want to
read them out how to handle the handle.
So I'll read them out to you. So the first one is are you creating and sharing
success plans and EBR is the same time and what cadence are using for both.
Yeah, great question so we for success planning we set a standard to have them
be generated quarterly.
And the benefit of them being automated now is we're actually seeing success
plans being generated more than once a quarter so as a customer starts working
through the recommendations that we have.
It's simple for a CSM to spin up a new set of recommendations based on the most
recent data for that account.
So we have a standard of quarterly for both success plans and EBRs.
And typically at the EBR is a good sort of opportunity to generate a new plan
and kind of get that in place for the executive team at that account so you
know we plan for quarterly but we've seen since we've automated it we've
actually see success planning
be it's about twice a quarter now which is really great to see.
Awesome.
Next one is can you expand on your process best practice around contact hygiene
slash maintenance that facilitates your customer journey campaigns.
Yeah, this is this has been a topic we were in the middle of cleaning it up
actually at the moment.
We for now we have a very simple construct in each account a contact either an
executive contact or a what we'll call CS contact meeting they're part of our
court team.
And that work when we were small, and now there were a lot bigger that's not
quite working so we're actually working through changing our contact management
to where each contact gets assigned a persona that they have within that
company within their domain
with with a different domains at a company.
And then also some different granger check to say hey this contact is on our
court team and they should get an NPS survey but you know they don't want
access to academy or we can kind of select kind of what we want them to get
access to. And so I think today we've learned that contact hygiene is actually really
critical to run these campaigns at scale.
And so that doing it at scale allowed us to see the problems we had and now we
've worked to clean that up so actually this in October we have like the whole
month is dedicated to like new contact management systems are launching a new
one today for references.
And then just making sure that that will give us the opportunity to actually
run more of these automated campaigns. So it's very important we learned the
hard way of kind of keeping the hygiene okay really limits us here so we're
investing more there.
And I think this question is super pertinent not just for what you presented
but for a lot of AI and that's how did you manage to justify the cost of the
internal AI and data science teams.
Or were they already in place and you just utilize them and I guess that's a
very important question nowadays on how to pay for AI and moving forward.
Yeah this is where getting data to justify resources is really critical I think
time tracking for us was just huge because it allowed us one to see where
people are spending their time.
We realize that CS and spending lots of time on debt creation, educating
customers.
And what this is you can apply it you know it needs pretty easy dollar value
you know how much you pay your CSM is all in.
You know definitely use all in pricing kind of you know their bonus plus plus
benefits and say this is a real cost of business and say look if we actually
reduce time by three hours if we think this investment in AI can say three
hours per CSM per week.
You can scale that out and have a pretty good ballpark estimate of how much
money you could save and use that to justify new spend.
So we use it in two ways one is any spin that we have because obviously you
know call notes they do cost us ultimately we have to call and GPT and there's
a cost to that.
But we're able to say well the cost is actually quite minimal relative to the
benefit we're getting from the business and so having that data was actually
really critical if we didn't have that call the data for time tracking.
I think it would have been a lot harder to justify and actually provide you
know real estimates for the expense so definitely if you don't have those
figure out ways to get those those estimates because those really where you're
going to be able to say look here's where we're at today.
Here's where we think AI will get us that delta right there is going to be your
time savings and where you can actually show impact so we'll definitely
recommend you come up with those kinds of constructs to report to the
leadership team to justify.
Both the you know if you have to buy something the expense of it but also the
resource that you need to use internally.
We do that's why the reporting for us and impact reporting and all this were
really important.
We weren't able to deliver that I don't think we would have been able to get
those resources and understandably so right you know I mean ultimately
resources are scarce for a reason and there's lots of things people could work
on at all times so for us this was our argument to the organization to say hey
we think this is super valuable and here's why.
And we just consistently make that argument over and over again.
Excellent.
The next one is where do you load the slide decks in the game site.
Is it just the URL or and how are they surfaced to CS to edit.
Yeah so we the we use an API call from that super blocks app to just send the I
think we do it via API just send the link over using the gains that API to a
table that we built in games.
And so that what we just have a simple table that shows the customer name the
date the report was generated the report name since they're different reports
you can auto generate and then the link to the Google sheet the Google slides.
And then inside that super blocks app that I showed you I can go back to it
real quick.
Where it says generated reports here in the middle. That's where the CSM would
get their link to the slides and those slides are.
We set the permissions where they get the slides they can edit them directly
some edit the the deck that we just generate for them.
Some make a copy of it and edit the copy and then post the copy someplace for
us to reference but that's how we allow them to edit them directly.
Excellent the next one I think is wrong customer education how do you measure
the courses impact and did it really help product adoption the education piece.
Yeah this has been really interesting I think we haven't made that shift yet
this is where initially we looked at just getting content in there, making sure
there is a robust set of content for customers to reference, and then making
sure we drove adoption of academy.
I think the impact pieces harder and that's been one of the things that we've
been working on building which is an understanding of if a customer takes
certain courses does that have impact.
In terms of as you mentioned product adoption or awareness.
So what we did at first is we've been developing those metrics have been a bit
harder to operationalize is just ask CSM's you know just pulled them casually
say hey do you feel like your customers are asking more sophisticated questions
or having better conversations. Or if you sent them a link to academy did you get any follow up questions or
did you feel at that really helped them and that gave us some qualitative data
in some indication that we were on the right track so we're still in the middle
of providing like a full fledged business
and you know what is the like in downstream business and customer impact, but
we you know sent some in terms surveys to help us understand a bit that we're
on the right track so it's I guess for education it's been a little bit harder.
And something that we are you know our education team is really focused on this
half spent a big ask for them and so, you know I think we're confident that
will be some good results based on our you know kind of interim surveying of CS
M's but something that
we've got to kind of operationalize at scale.
There's there's a couple questions on this one fell of how you tracked CSM time
and was it automated or was it manual to base out for port or do you have some
sort of reports that spend x amount of time doing this y amount of time doing
that. Yeah so we just did it in Google sheets we wanted to choose the path of these
resistance but didn't want to teach people to use a new tool we didn't want to
buy a configure tool.
So what we did is we.
I created we work with the CS in leadership team and we define the categories
of time tracking customer meeting internal meeting you know deck preparation
customer education internal enablement you know all those things that CSM to do
And then it was probably ended up being like 10 or so categories but didn't
want it to be huge we don't want to like 10 to 12 categories.
And then we built a sheet that just allowed a CSM they gave them a sheet each
week and said you know we pre populated the dates in it so that it was really
easy to fill out.
And then we use some data validation rules to allow you just to select you made
a copy of the sheet for yourself.
And then each you know row is basically the date, whatever activity it was the
customer was attributed to that allowed us to see which customers are actually
taking up the most time.
And then it allows you some side analysis I think one CSM leader that a whole
analysis of his team and kind of like understanding which customers took more
time and why that helped with resourcing.
And then the activity and then the time and we track time and 15 minute
increments.
And you just said hey this took you know a quarter of an hour half an hour one
hour two three four five six seven, you know whatever it was and then you just
submitted that sheet at the end of each week over to the ops team.
And then we created that data to report back our goal here wasn't until like in
analyze any individual CSM it was really to analyze what was happening at scale
which I think helped us kind of get people bought in this wasn't like a back
you know back in a way to track
whether you're doing work and odd is really more like hey we don't really care
individually what people are doing we care actually in aggregate what the team
is doing and how much time is taking and so we just went to Google sheet route
and create a simple template and that
was to just like remove any complexities of new systems and change management
and things like that.
Cool. How did you handle scale given customers the ability to opt in or out
from having their confidential data sent to a third party AI services and data
processors.
So we, we sent out the data processor like a third party to processing
announcement saying hey we've like, sent it out you know we use open AI but you
know because of all these privacy rules you know talk you know walk them
through basically the what we're you know open eyes on using the state of the train
their models and things like that now in that you get the ability to opt out.
There are certain even our product uses a lot of these large language models so
if you opt out or certain features or certain products you have that you know
don't work or are not going to work as well.
And we have alternative models we can use based on you know what geography you
're in or so on so forth but you know we just went through the kind of our legal
team really handled that and security team handled those communication with
customers and allowing them to opt out.
There's a quick generate lots of questions and so our security team leaned in
quite heavily to hop on calls with customers this is a whole thing and they go
over the love of the summer where our security team is having lots of
conversations with customers and just you know letting them know how this
work behind the scenes that they did it was secure that we had all the security
things in place there did it was a major train and all those things so
you know it was a whole process we went through but something we're very open
with our customers and willing to hop on calls and answer questions and that
really help I think smooth over any concerns people had
because we are an product and we this is you know we use these models the
normal course of our product.
A lot of this stuff does get vetted during the sales cycle so
you know the customer goes to a pretty extensive or we go through a pretty
extensive security with the customer so
we do benefit in some ways of these questions being answered in the sales
process and those approvals being done there but
I think we had to get on the phone with customers quite often this summer and
just you know walk them through what these changes meant and how everything was
still secure and up to par and the ways that they had already approved.
There's a couple more about data privacy like is it HIPAA is what you're doing
HIPAA compliant.
If you had a customer who had the follow HIPAA rules would this be okay or is
that a different set of permissions that you would have to get.
The product is like in terms of automation stuff we don't really deal with any
like HR private data that's all stripped out and we are good at science team
may have lots of masking so the data that we get in CS is already masked and
remove all this information so we don't
typically I don't even think the CSC is like a name of a user and like that we
just see you know user IDs at any any that data is mostly aggregated anyways at
our level.
And on the product side itself yeah there's all these permission that gets put
in place and any integration we have is put on rails so we're only getting the
data that we need is properly masked that's but that's one of our product and
implementation team side to manage
and so all of them each those compliance rules and we make sure that you'll see
us just doesn't have any access to any any information they don't need to have.
And we have one minute left so I'm going to combine the last two that we have
here the first one is any automation ideas for scheduling meetings out of your
AIPs and also can you explain real quick that it's been creating email
templates with different tokens to personalize the email versus the automated
emails generated.
Oh yeah so the I think what you're asking the second one is like the difference
in email assist and journey orchestrator and essence right and we use.
So what we do is if we feel we need CS to make any edits to the email, for
instance like responding to NPS surveys we actually want you to, you know, a CS
M to say in that response, you know reference to actual feedback they got in the
survey. And so that's perfect for email assist we can do things I tokenize the name and
those basic things so it is personalized but then also, it's like pretty clear
in the email template that you need to go put in some information here.
And so, anything that requires any edits by the CSM we use email assist for,
and that way they get 80% of the template done for them and they just have to
go on and type us an answer to.
So if it's more something that we don't need any CSM input in more like these
like on winning programs are more like hey, it's your first weeks is going live
You know, please review this documentation or Academy or community or a GOC
site and come to the next meeting prepared with these things.
That doesn't need any CSM inputs this is a great example of a journey orchestr
ator that we can do because you're really just sending something personalized to
the customer and letting them know what they should do to prepare for a meeting
and then that way, you know, they show up to a meeting, typically most of them
do the homework. And we can have more of both conversations so ultimately we kind of bifurcate
them based on do we need CSM to make any edits or can we just like run this
thing because it's more announcement driven or generalized and that's how we
differentiate between the two programs.
Your other questions about call scheduling correct.
Yes.
Yeah, so we're pretty high touch so I think most CSM is managed about 12 to 13
customers on the enterprise side.
I think with our commercial team which are managing more like 25 to 30.
We're looking into, you know, we have some count we have count on the accounts
for them and right now it's just standard in there.
You know, they can send their customers in a normal way.
However, we do use it quite often to do office hours scheduling so in Academy
and also there's a dedicated page for the commercial team I think it's an
academy as well where customers can just go and sign up for office hours and
then they get access to their their CSM or another person that's asking to help them out and
maybe it's not really cool model but just an easier way to contact people if
you need help, you know, at that moment.
But we haven't because we're high touch.
We don't really have a need to do it because we typically have like a weekly
cadence but in these scaled motions like Academy as well as commercial we're
using it more for like an office hours type approach and self service.
Motion for the customer to sign up at their own leisure. So that's kind of how
we're managing it there.
Awesome. Thank you, Philip. This was fantastic. I want to thank everybody for
joining us today. The recording will be up here soon. I know that they'll be
sharing the deck via PDF at some point here to all participants.
Philip have a great rest of your day and we'll talk soon. Appreciate it.
Awesome. Thank you so much everybody. Thank you for joining me. And I think any
unanswered questions, I believe I'll go in the community and answer them so we
'll definitely be in there to answer anything else you have and happy to want to
contact me directly on LinkedIn or through the community
please please feel free happy to some more time chatting with you all.
[Silence]
This is a test comment.
This is a longer test comment to see how this looks if the person decides to ramble a bit. So they're rambling and rambling and then they even lorem ipsum.