You're a PM at Kayak. How would you measure your company's success?

Introduction

Measuring Kayak's success as a travel site requires a comprehensive approach that considers multiple stakeholders and the complex dynamics of the online travel industry. To address this product success metrics challenge, I'll follow a structured framework covering core metrics, supporting indicators, and risk factors while considering all key stakeholders.

Step 1

Product Context

Kayak is a metasearch engine for travel, aggregating results from hundreds of travel sites to help users find the best deals on flights, hotels, car rentals, and vacation packages. Key stakeholders include:

1. Travelers (end-users)
2. Travel providers (airlines, hotels, car rental companies)
3. Online travel agencies (OTAs)
4. Advertisers
5. Kayak shareholders

The typical user flow involves:

1. Search: Users enter travel details (dates, destinations, etc.)
2. Results: Kayak displays aggregated options from various providers
3. Comparison: Users compare prices, amenities, and reviews
4. Booking: Users select their preferred option and complete the booking (often redirected to partner sites)

Kayak's strategy focuses on being the go-to platform for travel planning and booking, differentiating through comprehensive search results, price comparison tools, and user-friendly interfaces. Compared to competitors like Expedia or Booking.com, Kayak's metasearch model allows for a wider range of options but relies more heavily on partnerships for actual bookings.

As a mature product in the growth stage, Kayak's focus is on expanding market share, improving user engagement, and optimizing monetization strategies. Does this approach seem appropriate to move forward with?

Step 2

Goals

Step 3

North Star Metric

To align all stakeholders and guide decision-making, it's crucial to identify a single metric that encapsulates overall success. I propose "Gross Booking Value (GBV) per Active User" as Kayak's North Star Metric.

Definition: Total value of travel bookings made through Kayak (including partner redirects) divided by the number of monthly active users.

Calculation: GBV per Active User = Total Gross Booking Value / Monthly Active Users

This metric captures Kayak's success in driving valuable bookings while also considering user engagement. It aligns with the interests of all stakeholders:

• Travelers benefit from finding and booking desirable travel options
• Travel providers and OTAs gain from increased bookings
• Advertisers see value in an engaged user base
• Shareholders benefit from the revenue implications

For example, if Kayak's monthly GBV is $500 million with 10 million active users, the GBV per Active User would be $50. An upward trend in this metric would indicate that Kayak is successfully increasing the value it delivers to both users and partners.

Breakdown of North Star Metric

The North Star Metric can be broken down into its component parts:

Formula breakdown: GBV per Active User = f(GBV, MAU) GBV = f(Average Booking Value, Number of Bookings) MAU = f(User Acquisition, User Retention) Average Booking Value = f(Mix of Travel Products, Pricing and Deals) Number of Bookings = f(Search to Booking Conversion Rate, Repeat Bookings)

Step 4

Supporting Metrics

To operationalize and influence the North Star Metric, we need a set of supporting metrics. These metrics provide actionable insights into specific areas of performance, helping to fine-tune strategies and drive improvements.

Step 5

Guardrail Metrics

While focusing on growth, it’s essential to safeguard against unintended consequences. Guardrail metrics serve as checkpoints to ensure we maintain a balance between delivering value to users, partners, and stakeholders, while avoiding pitfalls.

Customer Satisfaction Score is crucial as it directly impacts user retention and word-of-mouth referrals. A drop below 4.5/5 could indicate issues with the user experience, potentially affecting the North Star Metric by reducing active users or booking frequency.

Click-through Rate to Partners is vital for maintaining strong relationships with travel providers and OTAs. If this falls below 15%, it could lead to partners reducing their inventory on Kayak, ultimately impacting the Gross Booking Value component of our North Star Metric.

Step 6

Trade-off Metrics

In optimizing Kayak’s performance, some decisions may involve trade-offs. By identifying trade-off metrics, we can monitor the balance between competing priorities, such as user experience and monetization, ensuring sustainable growth.

1. User Experience vs. Monetization

   • Metrics: Time to Book vs. Revenue per Session
   • Balance: Implement monetization features that add value to the user experience, such as premium filters or personalized recommendations

2. Breadth of Options vs. Search Speed

   • Metrics: Number of Search Results vs. Search Response Time
   • Balance: Implement efficient caching and prioritize most relevant results, with an option to load more

3. Partner Diversity vs. Booking Conversion Rate

   • Metrics: Number of Active Partners vs. Overall Conversion Rate
   • Balance: Continuously evaluate partner performance and optimize the mix to maintain a wide selection while prioritizing high-converting options

Step 7

Counter Metrics

As we scale, it's critical to proactively monitor for unintended negative outcomes. Counter metrics help identify risks or issues, such as pricing inaccuracies or increased support needs, and guide corrective actions to sustain trust and reliability.

1. Price Accuracy Rate

   • Purpose: Ensures the reliability of displayed prices
   • Calculation: (Accurate Price Displays / Total Price Displays) * 100
   • Action if low: Improve real-time data synchronization with partners, implement more frequent cache updates

2. Customer Support Contact Rate

   • Purpose: Monitors the frequency of issues requiring support
   • Calculation: (Support Contacts / Total Bookings) * 100
   • Action if high: Analyze common issues, improve self-service options, enhance booking flow clarity

3. Fraudulent Booking Rate

   • Purpose: Protects against financial and reputational risks
   • Calculation: (Fraudulent Bookings / Total Bookings) * 100
   • Action if increasing: Strengthen fraud detection algorithms, implement additional verification steps for high-risk bookings

Strategic Initiatives

1. AI-Powered Travel Recommendations

   • Rationale: Enhance user experience and increase booking value
   • Impact: Could improve GBV per Active User by suggesting higher-value or more relevant travel options
   • Challenges: Requires significant data analysis capabilities and potential privacy considerations

2. Expanded Loyalty Program

   • Rationale: Improve user retention and encourage repeat bookings
   • Impact: Could increase Monthly Active Users and Number of Bookings components of our North Star Metric
   • Challenges: Balancing program costs with user benefits, integrating with partner loyalty programs

Conclusion

As the travel industry evolves, Kayak's success metrics may need to adapt. Emerging technologies like virtual reality for hotel tours or blockchain for secure bookings could introduce new metrics. Additionally, shifts in travel patterns (e.g., increased focus on sustainable travel) might require adjusting our metric priorities to align with changing user values and industry trends.

Expand Your Perspective

• How might geopolitical events or global health crises impact our metric priorities?

• What role could user-generated content play in Kayak's success, and how would we measure its impact?

• How can we balance personalization with privacy concerns in our metrics approach?

Related Topics

• A/B testing strategies for travel platforms

• Integrating sustainability metrics into travel booking sites

• Balancing direct bookings vs. partner referrals in a metasearch model

• Leveraging big data for predictive travel analytics

• Mobile-first strategies for travel apps