Welcome to No Jitter’s Conversations in Collaboration series. In this current series we’re asking industry leaders to talk about how AI can boost productivity – and how we define what productivity even is – with the goal of helping those charged with evaluating and/or implementing Gen AI to gain a better sense of which technologies will best meet the needs of their organizations and customers.
In this conversation we spoke with Jackie Rocca, Vice President of Product at Slack, where she oversees the vision and execution of Slack AI, which brings generative AI natively and securely into Slack’s user experience.
Our conversation with Rocca was sparked by the stats cited in the first question below – tucked into the announcement of Slack’s AI availability was the pilot program finding that participants saved 97 minutes a week, and in further research, desk workers had spent up to 41% of their time on rote tasks. The latter statistic echoed one cited in our interview with Glean, that as “knowledge workers, we spend one third of our time just looking for information….”
So, in this conversation with Slack’s Rocca, we dive into how that 97 minutes of time savings was calculated and discuss some of the design philosophy that went into Slack AI. Part 2 of this conversation is with Christina Janzer, Slack's SVP of Research & Analytics and Head of Slack's Workforce Lab. Janzer provides some additional perspective on Slack’s research into desk worker productivity.
No Jitter (NJ): Slack AI became available recently. That news cited a finding from Slack’s pilot program where customers saved 97 minutes per user per week. Slack also released some survey data on the use of AI tools. So maybe we could start with the survey finding that desk workers spend 41% of their time on tasks that are “low value, repetitive or lack meaningful contribution to their core job functions.”
Jackie Rocca (JR): That was that was a shocking [stat], but I also get it. There's so much information that people have now. It wasn't long ago, we were all siloed in emails and if someone left the company or you weren't copied on the email, you had no idea what was happening. Now, we have access to all this information, but it can be hard to find what you're looking for to disseminate that information or just not get bogged down your entire day trying to catch up.
We do internal surveys every quarter to six months asking our customers about their biggest productivity pain points at work. A lot of it is centered around information overload or finding what they need to do their jobs which are some of the pain points that led us to build the features that we did for Slack AI. But our research team has also helped [define the real problems] – like spending 41% of your time on those lower value tasks. Just imagine if that time can be better spent with customers or being more creative or building new products.
NJ: Does Slack in those surveys have a working definition of productivity that is used in conversations internally or externally with clients or customers?
JR: I don't know if we have an official definition of productivity, but I'm happy to talk about it as we've thought through it for Slack AI. A lot of it comes down to time savings as one of the big things we're looking at. You mentioned the 97 minutes – that data was based on our pilot customers and testers. We were able to calculate, based on the messages that were summarized and the time spent on these tasks, the “before and after” of those [who] used the [summarization] feature.
We had a pilot period of a couple of months [that] started September/October 2023 and ran through February 2024, where we looked at how people were using Slack AI, and [its current] features around search and summarization – [e.g.,] helping users find information faster, catch up and disseminate conversations. So, rather than people reading every single message, [with Slack AI] they get a succinct summary of what they missed. For searches, we were able to track how those same cohorts were trying to navigate and [go] through messages to find what they were looking for, versus getting summarized highlights that directly answered their question.
I think what would be interesting, additional research is like how are people then using that time? If they save 97 minutes, are they spending [that] time in front of customers? What are the things that they're doing with that “replaced time,” so to speak? I think we'll continue to look at [this question] as the features develop and as people use [Slack AI] in new and different ways, as well.
NJ: During the pilot period you were able to actually quantify the time spent before and after?
JR: It's based on internal analysis. For example, “channel summaries” is one of the features we launched. In a channel context where there's a group having a conversation, and these can be fast moving conversations, so whether you are away for an hour or you’ve been out for a week, you come back and you [have to] read through everything to catch up. Now they're using Slack AI to summarize those conversations.
We [could] see the time that was spent when people were reading through those messages versus getting the summaries. And [with Slack AI], everything is cited and sourced so people can dig in more deeply if they want to. But we [could] get a sense of time spent “before and after” on how those features [were] used. We are building in analytics so customers will be able to see [their own usage before and after.]
NJ: All the other UCaaS platforms have generative AI built into them now. How does Slack AI work in conjunction with one of those platforms?
JR: Slack is a very open platform. We've seen a huge amount of interest of AI apps being built within Slack – more than 13,000 AI-powered apps that people have built and distributed in Slack. I think a lot of that is because Slack is already a conversational interface, and a lot of these experiences are conversational. It's also where work is happening – people using Slack are in it every day and it's one of their most heavily used work tools. We see Slack as becoming an “AI command center” because you can pull in [many different] tools and products so that you can access them directly from your flow of work.
NJ: Box is an example of one such integration. What does it mean that it is AI-powered – can I use Slack AI to search my data that's in Box?
JR: You can authenticate into Box and the Box AI experiences would be displayed in your flow of work … you can add them to a channel, you can “@mention” them, you can use it one-on-one from an app experience. Right now Slack AI is [only] indexing Slack content, but we are talking to customers because they are excited about adding more “corpus” to the native AI Slack experience.
NJ: I spoke with Glean earlier this year which provides an “over the top,” so to speak, enterprise search solution. You just mentioned how Slack AI enables search and summarization within an organization’s Slack corpus – what’s the difference here between Slack and a search solution like Glean?
JR: For Slack, it's in your flow of work. You don't have to add another tool or learn something new. We're listening to Slack customers [about] their pain points or where can we make the product experience better. So, yes, retrieving information and finding information is one of those pain points. But it's also things like summarization, and helping you save time in that way.
We're really focused on those native experiences within Slack that can drive productivity benefits from where you're already doing work. Glean is a great complement if you want to pull Glean into your Slack experience.
NJ: You mentioned in your blog post about not wanting to make folks prompt engineers. How do you make the distinction between the user needing to know how to talk to the AI as opposed to just talking to the AI? And how do you help ensure the user is getting back what they’re looking for?
JR: Googling something 15 years ago maybe wasn't as intuitive [back then] as it is today; maybe that’s what [using AI tools] will be like in the future. For where we're at with [AI] technology today, we wanted to help users not fail. We wanted to be as guided as possible and present AI in the places that you already would expect to search for something.
You saw that example on summarizing channels – you don't have to prompt it the right way: Summarize this channel, get three bullet points, include all the sources, do it for the last seven days, and do it in a professional tone. You don't need to figure out how to do that.
We have spent months trying to get to the best model for the job and the best prompt so that we take that work away from you. All you have to do is click summarize and you can pick the date range if you want it – since the last time you read [the thread] or the last seven days or pick your custom range. That's something that's much more intuitive and easier for users to grasp versus figuring that out on their own. Same with search. I showed you one [example] which included a natural language query, but we also present “question and answer” pairs based on the keywords [used].
One of our product principles at Slack is “Don't make me think,” and that's the way we are leaning into AI today. And again, that might change. It's very likely this becomes like Googling something a few years from now, but we really wanted to set up people for success, especially when they're trying to deploy this in a work setting.
NJ: How did current usage – or usage during the pilot period – inform product design, if at all?
JR: Out of our pilot program we did get a lot of great feedback that helped us refine the experience. For example, that “last seven days” option [I mentioned] wasn't there before, but [that was] a very common use case. We’ve been [focused on] prioritizing certain enhancements to make the experience better. We are also getting feature requests for things that are related to the features we built so we'll certainly be taking that feedback as well.
One thing that is really resonating is there's no training needed. People get Slack AI enabled and within the first five minutes of having it they're using it and getting value out of it. It's in their flow of work; they don't need to learn a new tool.
The second thing is we have an infrastructure model where we are hosting all these models within our [virtual private cloud] VPC, so none of the data is leaving Slack which is also attractive for our customers. We're not calling external APIs so the risk of data movement is largely mitigated because everything is within our infrastructure, which also I think is an advantage for these more native solutions that are within that boundary. So, customers are already comfortable with how their data is stored and being used in Slack and AI is no exception to that.
NJ: Which models do you use? You mentioned that there's a choice among models.
JR: We're not sharing [that] externally right now. I will say we are looking at the best model for the job. But the industry is constantly changing and we have a team that's constantly evaluating different models, so I'm sure the answer today will be different than the answer in the future as we try to figure out the best fit for [a] particular experience.
First and foremost is the data and security posture. We want to continue to hold models within our infrastructure – that’s critical to us – but also quality, speed, and opening up brand new use cases that maybe weren't possible a few months ago. It's an exciting space to be a part of, because almost every week new capabilities are available.
This link provides a PDF of Slack's security, privacy and architecture policies. This page provides additional information on those same topics.
During a short demo of Slack AI, Rocca made a few points:
The following is a short animation of Slack AI being used.
