Why has Optimizely's Feature Flags service seen a 20% drop in daily active users over the past month?

Introduction

Optimizely's Feature Flags service has experienced a significant 20% drop in daily active users over the past month, raising concerns about user engagement and product performance. To address this issue, I'll employ a systematic approach to identify, validate, and resolve the root cause while considering both immediate and long-term implications for the product.

Step 1

Clarifying Questions (3 minutes)

• Considering the timing, I'm wondering if there have been any recent product updates or changes to the Feature Flags service. Have there been any significant releases or modifications in the past 1-2 months?

Why it matters: Recent changes could directly impact user behavior and engagement. Expected answer: Information about recent updates or lack thereof. Impact on approach: If there were changes, we'd focus on their potential impact; if not, we'd look at other factors.

• Looking at the user base, I'm curious about the distribution of this 20% drop. Is it evenly spread across all user segments, or are certain groups more affected?

Why it matters: Helps identify if the issue is universal or specific to certain user types. Expected answer: Details on affected user segments. Impact on approach: Targeted investigation based on affected segments.

• Considering potential external factors, have there been any significant changes in the competitive landscape or market conditions for feature flag services recently?

Why it matters: External factors could influence user behavior independently of product issues. Expected answer: Information on market changes or competitor actions. Impact on approach: Adjust focus between internal and external factors based on the response.

• Regarding system performance, have there been any reported issues or increased error rates in the Feature Flags service over the past month?

Why it matters: Technical issues could directly impact user engagement and retention. Expected answer: Data on system performance and error rates. Impact on approach: If issues are present, prioritize technical investigation; if not, focus on other areas.

• Considering user feedback, have there been any notable changes in sentiment or specific complaints about the Feature Flags service in recent weeks?

Why it matters: Direct user feedback can provide valuable insights into potential issues. Expected answer: Summary of recent user feedback trends. Impact on approach: Focus on addressing specific user concerns if present, or look deeper into usage patterns if feedback is neutral.