Today, ServiceNow announced new generative AI (Gen AI) and governance capabilities for its Now Platform. These include new, expanded Now Assist capabilities that provide greater visibility and controls with AI Governance, as well as native multilingual support in Now Assist and new Now Assist use cases for configuration management, contract management, legal services, and health and safety.
Now Assist is ServiceNow’s Gen AI assistant which is embedded into ServiceNow’s products (which No Jitter have covered in multiple articles).
ServiceNow’s AI Governance new governance capabilities consist of three parts:
“We are providing a foundational AI inventory so you get a single place where you can view all the AI assets in your enterprise,” said Ravi Krishnamurthy, Vice President of Product Management, AI Platform and Responsible with ServiceNow. “On top of that, we are launching the Now Assist Data Kit, which is a way for you to evaluate skills and AI systems that you build on the platform, Now Assist Analytics which allows you to look at what’s happening in production and measure things like the value delivered and [if the AI] hallucinating, and Now Assist Guardian, which provides guardrails that you can put into your Gen AI pipelines.”
Krishnamurthy highlighted the technical complexity associated with deploying AI, generative or predictive, and emphasized how the platform-based approach, which ServiceNow has pursued, helps ameliorate that complexity by providing a ‘single pane of glass’ into an enterprise’s data, systems, applications, etc.
For example, announced on October 23, 2024, ServiceNow’s Workflow Data Fabric is a layer that facilitates connecting various data sources – structured, semi-structured and unstructured – so that AI can access that data and provide better assistance or, as AI Agents become more prevalent (and trustworthy), take action.
Krishnamurthy further highlighted the ‘people’ complexity associated with AI deployments – developers are involved, of course, but also employees in risk, legal, compliance, plus the users themselves, who get involved in different parts of the lifecycle.
“When I do executive briefings, the chief counsel often sits right next to the CTO and listens to what we have to say,” Krishnamurthy said. “That’s why we call AI a cross-functional team sport. It’s not like you ‘build AI’ and at the very end there’s a checklist. It doesn’t work that way – there’s a high degree of fluidity, ambiguity and iterations.”
Visibility into all AI and data assets is the first step in building a control/governance layer. There are the AI assistants and AI Agents, and the data sets that are used to either build the models or evaluate the systems or models – and then there are the relationships among those components. Good governance means, at least in part, knowing how those components interact and what those relationships are.
“If I come and ask you, ‘hey, you launched this chat bot, what did you test on? What data did you use? Which model did you use? Was that model responsibly sourced?’ I ask that because now there's third- and fourth parties involved that you also need to govern,” Krishnamurthy said. “These models come from somewhere and they get data sets from somewhere else. It's just a big supply chain you need to look at for risk management.”
Controls and governance occur at different levels, as well. For example, a company might have a policy against sharing data with LLM providers. Or a lower-level control for an HR chat bot might involve detecting sensitive topics and moving those interactions to a human rather than allow the bot to answer. Other controls might dictate where data should be processed per various regulations or how loan applications should be reviewed.
These controls – guardrails – need to be built in from the start and can be applied to ‘simple’ bots or more sophisticated AI Agents. And this is where ServiceNow’s platform approach helps.
“With Data Kit, you build very robust evaluation data sets and scenarios that you run through to make sure your agent stays within the boundaries you have set for it,” Krishnamurthy said. “Analytics is post deployment. You want to analyze it – and it will go wrong sometimes – but you want to make sure it doesn't go wrong in a terrible way. When it does, you need to immediately create a response. It’s like an incident change – which is what our platform is well suited to do.”
Hallucinations – i.e., when the AI produces obviously very incorrect or misleading results in response to a query – are one way Gen AI can go wrong. Detecting those hallucinations in tests or production involves four methods: explicit feedback, implicit feedback, auto-evaluation and manual spot checking. Krishnamurthy described explicit feedback as ‘thumbs up or down’ feedback from the user. Implicit feedback is more nuanced and reflects whether the user used the AI-generated response. Auto-evaluation is when a ‘judge’ model tests the AI output.
“Generally, you don't want [the AI evaluation] to be fully automated. You want some manual spot checking on top of it,” Krishnamurthy said. “Say the AI is hallucinating greater than 20% in an hour. Then I’d have [the human] step and check if it’s really happening. If it is, then the person can address it.”
With respect to assessing the performance of AI, Krishnamurthy focused on the value delivered by AI citing the post-call summarization use case that has become one of the easy-to-explain ‘wins’ for Gen AI in internal/external contact centers. “If you want to summarize an incident for an agent, that saves them time, and [maybe the AI] also creates a knowledge base article that deflects cases down the road. If you deflect an incident, that has value,” Krishnamurthy said. “Internally, we have seen almost $15 million annualized savings within six months of launching Now Assist, and this is with an initial set of 15 or 20 use cases. We expect to see more and more over time.”
There is follow-on value, as well, Krishnamurthy said. “CSAT might go up and you can start value modeling that, as well, and our initial ones are very conservative and [lean more toward] productivity gain type of value model. If you’re solving a case in two days less time on average than before, that matters in terms of your employee satisfaction but [that’s] a bit harder to model.”
