What metrics would you use to evaluate Moveworks's natural language understanding capabilities?

Introduction

Evaluating Moveworks's natural language understanding capabilities requires a comprehensive approach to product success metrics. To address this challenge effectively, I'll follow a structured framework that covers core metrics, supporting indicators, and risk factors while considering all key stakeholders. This approach will help us gain a holistic view of the product's performance and identify areas for improvement.

Step 1

Product Context

Moveworks is an AI-powered conversational platform designed to automate IT support and employee service tasks. Their natural language understanding (NLU) capabilities are crucial for interpreting and responding to user queries accurately and efficiently.

Key stakeholders include:

1. End-users (employees) seeking IT support
2. IT teams looking to reduce workload and improve efficiency
3. Company leadership interested in cost reduction and productivity gains

User flow:

1. Employee submits a query through chat or email
2. Moveworks NLU interprets the request
3. System attempts to resolve the issue or routes to appropriate human support

Moveworks' NLU capabilities are central to their value proposition, differentiating them from traditional IT helpdesk solutions. The product is in the growth stage, with increasing adoption across various industries.

Competitors like Aisera and Espressive also offer AI-powered IT support, but Moveworks claims superior NLU accuracy and resolution rates.