You're a PM at Kayak. How would you measure your company's success?

Introduction

To approach this product success metric problem effectively, I will follow a simple product success metric framework. I'll follow a structured framework that covers core metrics, supporting indicators, and risk factors while considering all key stakeholders.

Step 1

Product Context

Kayak is a metasearch engine and travel aggregator that helps users find and compare travel options across multiple providers. The platform allows travelers to search for flights, hotels, rental cars, cruises, and vacation packages from hundreds of travel sites simultaneously.

Key Stakeholders:

• Travelers/Users: Looking for the best travel deals and a seamless booking experience
• Travel Providers: Airlines, hotels, car rental companies seeking customer acquisition
• Advertisers: Travel-related businesses wanting visibility to potential travelers
• Investors/Shareholders: Expecting growth and profitability
• Employees: Needing clear direction and purpose

User Flow:

1. Search: Users enter travel parameters (dates, locations, preferences)
2. Compare: Users view aggregated results from multiple providers with filtering options
3. Select: Users choose preferred options based on price, convenience, and features
4. Redirect/Book: Users are directed to provider sites or complete booking through Kayak

Strategic Fit: Kayak's core value proposition is saving travelers time and money by aggregating options across the fragmented travel market. This aligns with the broader strategy of empowering travelers with comprehensive information and choice while generating revenue through referrals, advertising, and booking fees.

Competitive Landscape: Kayak competes with other metasearch engines like Skyscanner and Google Flights, as well as OTAs like Expedia and Booking.com. Kayak differentiates through its powerful filtering capabilities, price prediction tools, and clean user interface that simplifies complex travel decisions.

Product Lifecycle Stage: Kayak is in the maturity stage, with established market presence and brand recognition. The focus is on optimizing the core product while expanding into adjacent travel services and international markets to maintain growth.

Step 2

Goals

Step 3

North Star Metric

For Kayak, I propose Monthly Booking Value (MBV) as our North Star Metric. This represents the total monetary value of all bookings facilitated through Kayak in a given month.

Definition: MBV = Sum of all booking values (in USD) that originated from Kayak searches and resulted in completed transactions with partners.

Why this metric captures success: MBV holistically captures Kayak's ability to:

1. Attract users to the platform (acquisition)
2. Help them find relevant travel options (engagement)
3. Provide compelling enough information to drive booking decisions (conversion)
4. Generate revenue for the business (monetization)

Stakeholder value alignment:

• Users: Higher MBV indicates users are finding valuable travel options
• Travel Providers: Higher MBV means more bookings for partners
• Advertisers: Higher MBV correlates with more eyeballs and potential customers
• Investors: Higher MBV directly relates to revenue and growth potential
• Employees: Clear indicator of company success and potential bonuses/growth

Hypothetical data: If our MBV is $500M in January and grows to $550M in February (+10%), this indicates strong performance. If it drops to $450M in March (-18%), we'd need to investigate seasonal factors, competitive pressures, or product issues.

Breakdown North Star Metric

MBV can be broken down into its component parts using the following formula: MBV = MAU × Search Rate × Booking Rate × Average Booking Value

• Monthly Active Users (MAU): Number of unique users who visit Kayak in a month

  • MAU = f(New Users, Returning Users)
  • New Users = f(Marketing Effectiveness, Brand Awareness)
  • Returning Users = f(Retention Rate, User Satisfaction)

• Search Rate: Average number of searches per active user

  • Search Rate = f(UI Usability, Search Functionality, User Intent)

• Booking Rate: Percentage of searches that result in bookings

  • Booking Rate = f(Click-through Rate, Partner Conversion Rate)
  • Click-through Rate = f(Result Relevance, Price Competitiveness)
  • Partner Conversion Rate = f(Handoff Experience, Partner Site Experience)

• Average Booking Value: Average dollar amount of each booking

  • Average Booking Value = f(Product Mix, Seasonal Factors, Economic Conditions)

Step 4

Supporting Metrics

Step 5

Guardrail Metrics

User Satisfaction Score: This metric captures how users feel about their experience with Kayak. It's measured through in-app surveys and feedback forms. If this drops below our threshold, it could indicate UX issues, missing features, or poor search results that would eventually impact our North Star Metric as users abandon the platform.

Partner Retention Rate: This measures the percentage of travel providers who continue their partnership with Kayak year over year. A high retention rate indicates that our platform is delivering value to partners through quality referrals. If this drops, we risk losing inventory comprehensiveness, which would directly impact our ability to provide users with the best options.

Profit Margin: While driving booking value is important, we must ensure we're doing so profitably. This metric ensures we're not sacrificing financial health for growth. If margins drop below threshold, we may need to optimize our monetization strategy or reduce operational costs.

Site Reliability: Technical performance is critical for a search platform. Users expect instant results and seamless experiences. Poor reliability leads to abandoned searches and damaged trust, directly impacting our North Star Metric.

Brand Sentiment: This measures public perception of Kayak through social listening and review analysis. A strong brand drives organic acquisition and loyalty, while negative sentiment can increase acquisition costs and reduce retention.

Step 6

Trade-off Metrics

1. Comprehensiveness vs. Speed

• Trade-off: More comprehensive search results (including more partners) can increase load times
• Metrics in tension: Number of partners integrated vs. Average search response time
• Balancing strategy: Implement progressive loading, where initial results appear quickly while additional options load in the background; prioritize most relevant results first

2. User Experience vs. Monetization

• Trade-off: Aggressive monetization (ads, sponsored listings) can degrade user experience
• Metrics in tension: Ad revenue per user vs. User satisfaction score
• Balancing strategy: Implement clear labeling for sponsored content, maintain quality standards for ads, and use personalization to ensure relevance of commercial content

3. Partner Prominence vs. User Value

• Trade-off: Partners who pay more want better placement, but users want truly best options
• Metrics in tension: Partner revenue vs. Booking conversion rate
• Balancing strategy: Use a blended ranking algorithm that considers both partner value and user relevance; provide transparency to users about how results are ranked

4. Short-term Revenue vs. Long-term Growth

• Trade-off: Maximizing immediate revenue might sacrifice future growth potential
• Metrics in tension: Quarterly revenue vs. User retention rate
• Balancing strategy: Set revenue targets that account for sustainable growth; invest portion of current profits into features that drive long-term retention

Step 7

Counter Metrics

1. Search Abandonment Rate

• Purpose: Identifies when users start but don't complete searches
• Avoiding pitfalls: Prevents overemphasis on total search volume that might hide usability issues
• Actions if problematic: Analyze drop-off points in the search flow, improve auto-suggestion, simplify input requirements

2. Price Competitiveness Index

• Purpose: Measures how Kayak's displayed prices compare to direct booking and competitors
• Avoiding pitfalls: Ensures we're not driving volume at the expense of offering competitive prices
• Actions if problematic: Renegotiate partner agreements, improve price scraping technology, implement price matching features

3. Feature Bloat Score

• Purpose: Tracks complexity of the interface as features are added
• Avoiding pitfalls: Prevents degradation of the core experience through excessive feature addition
• Actions if problematic: Conduct feature pruning exercises, simplify UI, implement progressive disclosure of advanced features

Strategic Initiatives

Based on the metrics framework outlined, I would propose these strategic initiatives:

1. Personalized Travel Assistant

• Rationale: Users are overwhelmed by choices; personalization can increase conversion
• Impact on metrics: Should improve booking rate and repeat usage rate
• Implementation challenges: Requires sophisticated data infrastructure, privacy considerations, and machine learning capabilities
• Approach: Start with simple preference-based recommendations, evolve to predictive suggestions based on past behavior and similar travelers

2. Seamless Multi-modal Journey Planning

• Rationale: Travelers increasingly want end-to-end journey planning across transportation modes
• Impact on metrics: Could increase average booking value and expand addressable market
• Implementation challenges: Complex integration with various transportation providers, solving for connection reliability
• Approach: Begin with major routes and transportation hubs, expand based on usage patterns

3. Price Guarantee Program

• Rationale: Price confidence is a major factor in booking decisions
• Impact on metrics: Should improve booking conversion rate and brand sentiment
• Implementation challenges: Financial risk, technical implementation of price monitoring
• Approach: Start with limited scope (e.g., flights only), set reasonable conditions, and expand based on ROI

Conclusion

As travel technology evolves, Kayak's success metrics will need to adapt. Emerging technologies like AI-powered recommendations, voice search, and augmented reality for hotel visualization may create new opportunities and metrics to track. Additionally, changing travel patterns post-pandemic and increasing focus on sustainable travel options may shift what users value most.

The framework I've outlined provides a comprehensive view of Kayak's success while remaining adaptable to these changes. By focusing on Monthly Booking Value as our North Star while monitoring supporting and guardrail metrics, we can ensure balanced growth that serves all stakeholders in the travel ecosystem.

Expand Your Perspective

• How might blockchain technology impact trust and verification in travel bookings, and what new metrics might emerge?

• What role could Kayak play in the growing "workation" trend, and how would we measure success in that space?

• How might climate consciousness change travel decision factors, and how should our metrics evolve?

Related Topics

• Travel industry vertical expansion strategies

• Mobile-first product development for travel platforms

• Data privacy considerations in personalized travel recommendations

• Marketplace dynamics in two-sided travel platforms

• Loyalty program effectiveness in metasearch environments