What caused the sudden 30% decrease in email click-through rates for Zulily's flash sale notifications last week?

Introduction

The sudden 30% decrease in email click-through rates for Zulily's flash sale notifications last week is a critical issue that demands immediate attention. This significant drop could have far-reaching implications for the company's revenue and customer engagement. I'll approach this problem systematically, focusing on identifying the root cause, validating hypotheses, and developing both short-term and long-term solutions.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might have been a recent change in email content or design. Has there been any modification to the email template or content strategy in the past two weeks?

Why it matters: Changes in email content can significantly impact click-through rates. Expected answer: Yes, we updated our email template last week. Impact on approach: If confirmed, we'd focus on A/B testing the new vs. old template.

• Considering user segments, I'm curious about the distribution of this decrease. Is the 30% drop consistent across all user segments, or are some groups more affected than others?

Why it matters: Identifying affected segments can help pinpoint specific issues. Expected answer: The decrease is more pronounced in our newer customers. Impact on approach: We'd investigate onboarding processes and segment-specific content.

• Given the flash sale nature, I'm wondering about the timing of these emails. Has there been any change in the send time or frequency of these notifications recently?

Why it matters: Email timing can greatly affect open and click-through rates. Expected answer: We've been experimenting with different send times. Impact on approach: We'd analyze performance across different time slots and adjust accordingly.

• Thinking about external factors, I'm curious if there have been any significant competitor actions or market changes in the past week that could influence customer behavior?

Why it matters: External factors can sometimes explain sudden metric changes. Expected answer: No major competitor actions, but there was a big sale event in the retail industry. Impact on approach: We'd consider how to differentiate our offerings and messaging.