Why has Collective Health's member portal login rate dropped by 15% over the past month?

Introduction

The recent 15% drop in Collective Health's member portal login rate over the past month is a concerning trend that requires immediate attention. As we analyze this issue, we'll follow a systematic approach to identify, validate, and address the root cause while considering both short-term fixes and long-term implications for the product.

I'll begin by asking clarifying questions to gather essential context, then rule out basic external factors before diving deep into the product ecosystem and user journey. We'll break down the metric, gather relevant data, and form data-driven hypotheses. Through root cause analysis and validation, we'll develop a comprehensive plan to resolve the issue and prevent future occurrences.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might have been a recent product update. Has there been any significant change to the member portal in the last 1-2 months?

Why it matters: Recent changes could directly impact user behavior. Expected answer: Yes, there was a UI refresh. Impact on approach: If yes, we'd focus on the changes and user adaptation.

• Considering user segments, I'm curious about the distribution of the login drop. Is the 15% decrease uniform across all user groups, or are certain segments more affected?

Why it matters: Helps identify if the issue is global or specific to certain users. Expected answer: The drop is more pronounced in older users. Impact on approach: If segmented, we'd tailor solutions to specific user groups.

• Given the nature of health insurance, I'm wondering about seasonality. How does this month's login rate compare to the same month last year?

Why it matters: Rules out normal seasonal fluctuations. Expected answer: This is an unusual drop compared to last year. Impact on approach: If seasonal, we'd focus on temporary factors; if not, we'd look at recent changes.

• Thinking about potential technical issues, has there been any change in the systems or methods used to measure login rates?

Why it matters: Ensures we're comparing apples to apples. Expected answer: No changes in measurement systems. Impact on approach: If changed, we'd need to validate the new measurement system.