Why has Handshake's job application completion rate dropped by 15% over the past month?

Introduction

The recent 15% drop in Handshake's job application completion rate over the past month is a critical issue that demands immediate attention. As we analyze this product challenge, we'll employ a systematic framework to identify, validate, and address the root cause while considering both short-term fixes and long-term strategic implications.

Our approach will involve a thorough examination of the problem, generation of data-driven hypotheses, and development of a comprehensive action plan. We'll start by clarifying the context, then rule out external factors before diving deep into the product's user journey, metric breakdown, and potential internal causes.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might be a seasonal factor at play. Has there been any change in the types of jobs or internships available on the platform recently?

Why it matters: Seasonal variations could explain the drop and influence our solution approach. Expected answer: There's been a slight decrease in summer internship postings. Impact on approach: If confirmed, we'd need to adjust for seasonality in our analysis.

• Considering user segments, I'm curious about the distribution of this drop. Is the 15% decrease uniform across all user types, or are certain groups more affected?

Why it matters: Identifying specific affected segments could point to targeted issues. Expected answer: The drop is more pronounced among undergraduate users. Impact on approach: We'd focus our investigation on undergraduate-specific features or content.

• Thinking about recent changes, have there been any updates to the application process or UI in the last 1-2 months?

Why it matters: Recent changes could directly impact user behavior and completion rates. Expected answer: A minor UI update was rolled out to the application form three weeks ago. Impact on approach: We'd prioritize investigating the impact of this UI change on user experience.

• Regarding data integrity, has there been any change in how we're tracking or calculating the job application completion rate?

Why it matters: Ensures we're not chasing a phantom problem due to measurement issues. Expected answer: No changes to tracking or calculation methods. Impact on approach: We can confidently focus on actual user behavior and product issues.