How would you measure the success of the AI customer-service assistant in the Bajaj Finserv app?

Introduction

Bajaj Finance has made AI central to its operating model under a programme it calls FINAI. Among the earliest visible pieces are conversational AI bots that answer customer queries on products such as the Insta EMI Card, personal loans, extended warranty, and insurance, inside the Bajaj Finserv app and on other service channels. Company disclosures in FY26 described AI handling a large majority of service resolutions and a sharp fall in human-handled service requests.

The question: how would you measure the success of the AI customer-service assistant in the Bajaj Finserv app? This is tricky because the easy metric (automation rate) can look great while customers quietly get worse service. I'll set product context, define goals, choose a North Star, and then add supporting, guardrail, trade-off, and counter metrics.

Step 1

Product Context (4 minutes)

The assistant sits in the "Help" area of the Bajaj Finserv app and on WhatsApp. It answers in English and several Indian languages, by text and increasingly by voice. Typical intents:

• "What is my next EMI amount and date?"
• "Why was I charged a bounce fee?"
• "Send my loan statement / NOC / interest certificate."
• "How do I block or unblock my Insta EMI Card?"
• "How do I claim on my extended warranty?"

Key stakeholders:

• Customers: Want fast, correct answers without waiting for a call centre.
• Bajaj Finance service and operations: Want lower cost per resolution and fewer repeat contacts.
• Collections and risk: Need sensitive conversations (bounces, overdue amounts) handled accurately and respectfully.
• Compliance: RBI's grievance-redressal and fair-practice codes apply; customers must be able to reach a human and escalate.

User flow:

1. Customer opens Help or messages on WhatsApp.
2. Assistant identifies the customer (logged-in session or OTP).
3. Intent detected; assistant fetches account data and answers or performs an action.
4. If unresolved, handoff to a human agent with the conversation summary.
5. Optional CSAT rating at the end.

Lifecycle stage: Scaling. The bot is live across major products; the question now is quality at scale.