what metrics would you use to evaluate intercom's chatbot automation system?

Introduction

Evaluating Intercom's chatbot automation system requires a comprehensive approach to product success metrics. To address this challenge effectively, I'll follow a structured framework that covers core metrics, supporting indicators, and risk factors while considering all key stakeholders. This approach will help us gain a holistic understanding of the chatbot's performance and impact.

Step 1

Product Context

Intercom's chatbot automation system is a sophisticated AI-powered tool designed to handle customer inquiries and support requests without human intervention. It's a critical component of Intercom's customer communication platform, aimed at improving efficiency and scalability for businesses.

Key stakeholders include:

1. Businesses using Intercom (primary customers)
2. End-users interacting with the chatbot
3. Intercom's product and engineering teams
4. Intercom's sales and customer success teams

The user flow typically involves:

1. An end-user initiates a conversation on a business's website or app.
2. The chatbot engages, attempting to understand and resolve the query.
3. If successful, the issue is resolved; if not, the conversation is escalated to a human agent.

This product fits into Intercom's broader strategy of providing comprehensive, AI-enhanced customer communication solutions. It competes with other chatbot providers like Drift and Zendesk, differentiating itself through deep integration with Intercom's other tools and its focus on conversational AI.

The chatbot automation system is in the growth stage of its product lifecycle, with ongoing improvements in AI capabilities and expanding use cases.