What caused the sudden 30% decrease in new user signups for Amplitude's mobile SDK yesterday?

Introduction

A sudden 30% decrease in new user signups for Amplitude's mobile SDK is a critical issue that demands immediate attention and thorough analysis. This significant drop in a key performance indicator could have far-reaching implications for the product's growth and overall business health. To address this problem, I'll employ a systematic approach to identify, validate, and resolve the root cause while considering both short-term fixes and long-term strategic implications.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking this could be related to a recent release. Has there been any significant update to the SDK or related systems in the past week?

Why it matters: Recent changes often correlate with sudden metric shifts. Expected answer: Yes, there was a minor update three days ago. Impact on approach: If confirmed, I'd focus on changes in that release.

• Given the specificity of the 30% drop, I'm curious about the typical fluctuation in this metric. What's the normal day-to-day or week-to-week variance in new user signups?

Why it matters: Understanding normal variance helps determine if this is truly anomalous. Expected answer: Usually within 5-10% week-over-week. Impact on approach: A 30% drop outside normal variance would indicate a significant issue.

• Considering the mobile SDK's integration process, I'm wondering about the distribution of affected users. Is this decrease uniform across all app categories or concentrated in specific types of applications?

Why it matters: Helps identify if the issue is global or specific to certain integrations. Expected answer: The decrease is more pronounced in gaming apps. Impact on approach: Would focus on investigating gaming-specific integration issues.

• Thinking about potential data discrepancies, has there been any change in how new user signups are tracked or reported in the last few days?

Why it matters: Ensures the observed decrease is real and not a reporting artifact. Expected answer: No changes to tracking or reporting systems. Impact on approach: Would rule out data collection issues and focus on actual user behavior or technical problems.