What caused the sudden 30% decrease in new user signups for Tally's credit card payoff feature last week?

Introduction

A sudden 30% decrease in new user signups for Tally's credit card payoff feature last week is a critical issue that demands immediate attention. This significant drop in a key performance metric could have far-reaching implications for user acquisition, revenue, and overall product health. To address this problem, I'll employ a systematic approach to identify, validate, and resolve the root cause while considering both short-term fixes and long-term strategic implications.

Step 1

Clarifying Questions (3 minutes)

• Given the abrupt nature of the decline, I'm wondering about recent changes. Have there been any updates to the app, marketing campaigns, or onboarding flow in the past two weeks?

Why it matters: Recent changes often correlate with sudden metric shifts. Expected answer: Yes, we deployed a new onboarding flow last Tuesday. Impact on approach: If confirmed, I'd focus on analyzing the new onboarding process first.

• Considering user segments, I'm curious about the distribution of the decline. Is the 30% decrease uniform across all user segments, or is it more pronounced in specific groups?

Why it matters: Identifying affected segments can narrow down potential causes. Expected answer: The decline is more significant among iOS users, around 45%. Impact on approach: I'd prioritize investigating iOS-specific issues or changes.

• Thinking about external factors, has there been any significant news or events in the credit card industry that might influence user behavior?

Why it matters: External events can sometimes explain sudden changes in user behavior. Expected answer: No major industry news, but a competitor launched a similar feature last month. Impact on approach: I'd consider competitive analysis as part of the investigation.

• Regarding data integrity, I'm wondering about our tracking systems. Have there been any changes or issues with our analytics tools or tracking implementation recently?

Why it matters: Ensures we're not chasing a phantom problem due to faulty data. Expected answer: No known issues, but we haven't audited the system recently. Impact on approach: I'd include a quick data integrity check in our investigation plan.