How would you measure the success of Red, UBS's in-house generative AI assistant for employees?

Introduction

Red is UBS's in-house generative AI assistant. UBS reported rolling it out to more than 85,000 employees during 2025, alongside Microsoft 365 Copilot for all staff, and its annual report counted hundreds of live AI use cases by year end. Red helps employees search internal knowledge, summarise documents, draft content and answer policy questions inside UBS's controlled environment. The prompt asks how we would measure Red's success. This is an internal-product metrics question, which tests whether candidates can go beyond "number of users" and link usage to productivity, quality and risk.

Step 1

Product Context (3 minutes)

Red is an enterprise AI assistant built on large language models, grounded on UBS content such as policies, procedures, product information and research, and integrated with the employee's tools.

Key stakeholders:

• Employees: advisors, operations staff, engineers in Pune, Hyderabad, Zurich and the US, compliance officers and support teams.
• Business heads: want productivity and better client service.
• Group Technology and the Chief Data Office: run the platform and models.
• Risk, compliance and data protection: ensure no leakage of client-identifying data and correct use.
• Regulators: FINMA and others watch model risk and outsourcing.

Typical user flows:

1. An operations analyst asks Red how to process an unusual payment type and gets the procedure with citations.
2. A client advisor asks Red to summarise CIO research on Swiss equities into three bullet points.
3. An engineer asks Red about internal coding standards or an API.

Lifecycle stage: Scaling. Distribution is done; the challenge is turning access into habitual, valuable use.