Why has edX's mobile app user retention decreased by 20% over the past quarter?

Introduction

The recent 20% decrease in edX's mobile app user retention over the past quarter is a significant issue that requires immediate attention. As we analyze this product challenge, we'll follow a systematic framework to identify, validate, and address the root cause while considering both immediate and long-term implications.

I'll approach this problem by first clarifying key details, ruling out external factors, and then diving deep into the product ecosystem, user journey, and relevant metrics. We'll generate data-driven hypotheses, conduct root cause analysis, and propose validation methods and solutions.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might be seasonality at play. Has this 20% decrease been compared to the same quarter last year?

Why it matters: Seasonal fluctuations could explain the retention drop and impact our solution approach. Expected answer: Yes, it's been compared and the decrease is still significant. Impact on approach: If seasonal, we'd focus on year-over-year trends; if not, we'd investigate recent changes.

• Considering user segments, I'm curious about the retention patterns. Are we seeing this decrease across all user types or is it more pronounced in specific segments?

Why it matters: Different segments may require tailored solutions. Expected answer: The decrease is more significant among casual learners. Impact on approach: We'd focus on understanding and addressing casual learners' needs specifically.

• Thinking about recent changes, have there been any major updates to the app or course offerings in the past quarter?

Why it matters: Recent changes could directly impact user behavior and retention. Expected answer: A new UI was rolled out two months ago. Impact on approach: We'd investigate the UI change's impact on user experience and engagement.

• Regarding the retention metric itself, has the definition or measurement method changed recently?

Why it matters: Changes in metric definition could lead to misinterpretation of data. Expected answer: No changes in definition or measurement. Impact on approach: We'd focus on actual user behavior changes rather than metric inconsistencies.