Why has customer satisfaction with Lemonade's AI-powered claims bot Maya decreased from 90% to 75% in the last quarter?

Introduction

The recent decline in customer satisfaction with Lemonade's AI-powered claims bot Maya from 90% to 75% in the last quarter is a critical issue that demands immediate attention. This significant drop could have far-reaching implications for customer retention, brand reputation, and overall business performance. To address this problem, I'll employ a systematic approach to identify, validate, and resolve the root cause while considering both short-term fixes and long-term strategic implications.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might have been recent changes to Maya. Have there been any updates or modifications to the AI model or user interface in the last quarter?

Why it matters: Recent changes could directly impact user experience and satisfaction. Expected answer: Yes, there were updates to the AI model. Impact on approach: If confirmed, we'd focus on analyzing the specific changes and their effects.

• Considering user segments, I'm curious about the distribution of the satisfaction decrease. Is the 15% drop consistent across all user groups, or are certain segments more affected?

Why it matters: Identifying specific affected segments could pinpoint targeted issues. Expected answer: The drop is more pronounced in certain user groups. Impact on approach: We'd prioritize investigating those specific segments and their unique characteristics.

• Thinking about external factors, has there been any significant change in the types of claims being processed by Maya in the last quarter?

Why it matters: Changes in claim complexity could affect Maya's performance and user satisfaction. Expected answer: There's been an increase in complex claims. Impact on approach: We'd focus on Maya's ability to handle more complex scenarios and potential improvements needed.

• Considering the measurement itself, has there been any change in how customer satisfaction is measured or calculated?

Why it matters: Changes in measurement could explain the apparent decrease without reflecting actual user sentiment. Expected answer: No changes in measurement methodology. Impact on approach: We'd focus on actual performance issues rather than measurement discrepancies.