What caused the sudden 20% drop in Onfido's document verification success rates for UK driver's licenses last week?

Introduction

The sudden 20% drop in Onfido's document verification success rates for UK driver's licenses last week is a critical issue that demands immediate attention. This analysis will systematically identify, validate, and address the root cause while considering both short-term fixes and long-term strategic implications.

I'll approach this problem by first clarifying key details, ruling out external factors, and then diving deep into the product ecosystem, metric breakdown, and data analysis. From there, I'll generate and validate hypotheses, conduct root cause analysis, and propose a comprehensive resolution plan.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking this could be related to a recent system update. Has there been any software deployment or changes to the verification algorithm in the past two weeks?

Why it matters: Recent changes often correlate with sudden performance shifts. Expected answer: Yes, there was a minor update to the image processing algorithm. Impact on approach: If confirmed, I'd focus on the technical aspects of the recent change.

• Considering the specificity to UK driver's licenses, I'm wondering about potential regulatory changes. Have there been any recent updates to UK driver's license formats or security features?

Why it matters: Regulatory changes can significantly impact document verification systems. Expected answer: No recent changes to UK driver's license formats have been announced. Impact on approach: If confirmed, I'd shift focus to internal factors or data anomalies.

• Given the magnitude of the drop, I'm curious about the distribution across user segments. Is this 20% drop consistent across all user demographics, or is it more pronounced in specific groups?

Why it matters: Uneven distribution could point to specific user experience issues or data biases. Expected answer: The drop is more significant among users aged 18-25. Impact on approach: If confirmed, I'd investigate factors specific to younger users' verification process.

• Considering potential data integrity issues, I'm wondering about our measurement systems. Have there been any changes to how we calculate or report verification success rates?

Why it matters: Metric definition or reporting changes can create false alarms. Expected answer: No changes to metric calculations or reporting systems. Impact on approach: If confirmed, I'd focus on actual performance issues rather than measurement anomalies.