What's causing the sudden 30% increase in failed identity verifications for new user sign-ups on Credit Karma?

Introduction

The sudden 30% increase in failed identity verifications for new user sign-ups on Credit Karma is a critical issue that demands immediate attention. This problem directly impacts user acquisition and could have far-reaching consequences for the platform's growth and user trust. I'll approach this analysis systematically, focusing on identifying potential root causes, validating hypotheses, and developing both short-term fixes and long-term solutions.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking this could be a recent change. When exactly did we start observing this 30% increase?

Why it matters: Pinpointing the timeframe helps correlate the issue with potential changes or events. Expected answer: Within the last week or two. Impact on approach: A sudden change suggests a specific trigger rather than a gradual trend.

• Considering user segments, I'm curious if this affects all users equally. Are we seeing any patterns in terms of user demographics or device types more affected by these failed verifications?

Why it matters: Identifying patterns could point to specific user groups or technical issues. Expected answer: Mobile users or younger demographics might be more affected. Impact on approach: This would focus our investigation on particular user segments or technologies.

• Given the nature of identity verification, I'm wondering about our data sources. Have there been any changes to our third-party data providers or APIs used in the verification process?

Why it matters: External data source changes could directly impact verification success rates. Expected answer: No recent changes, but we'll need to double-check. Impact on approach: If confirmed, we'd need to investigate data quality or API performance issues.

• Thinking about recent updates, have we rolled out any changes to the sign-up flow or identity verification process in the past month?

Why it matters: Recent changes could have unintended consequences on the verification process. Expected answer: A minor UI update was implemented two weeks ago. Impact on approach: We'd need to scrutinize this update and its potential impact on user behavior or system performance.