Why have course completion rates for Udemy's mobile app dropped 15% in the past month?

Introduction

The recent 15% drop in course completion rates for Udemy's mobile app is a critical issue that demands immediate attention. This decline not only impacts user satisfaction but also has potential long-term consequences for Udemy's business model and market position. To address this problem, I'll employ a systematic approach to identify, validate, and resolve the root cause while considering both short-term fixes and long-term strategic implications.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might be a recent app update. Has there been any significant changes to the mobile app in the past month?

Why it matters: Recent changes could directly impact user experience and course completion rates. Expected answer: Yes, there was a major UI overhaul. Impact on approach: If confirmed, we'd focus on UI/UX issues in our analysis.

• Considering user segments, I'm curious about the distribution of this decline. Is the 15% drop consistent across all user groups or concentrated in specific segments?

Why it matters: Identifying affected segments helps narrow down potential causes. Expected answer: The decline is more pronounced among new users. Impact on approach: We'd investigate onboarding processes and first-time user experience.

• Thinking about the course completion metric, I'm wondering about its definition. Has there been any change in how course completion is measured or tracked in the past month?

Why it matters: Changes in metric definition could explain the apparent decline. Expected answer: No changes in metric definition. Impact on approach: We'd focus on actual user behavior changes rather than measurement issues.

• Considering external factors, I'm curious about any major marketing campaigns or promotions. Have there been any significant changes in user acquisition strategies recently?

Why it matters: New user influx could affect overall completion rates. Expected answer: A recent promotion brought in a large number of new users. Impact on approach: We'd analyze how new user behavior differs from existing users.