Introduction
Harvey Agents are designed to execute legal work end to end: they plan steps, read documents, draft outputs and check their own work. A key product decision is how often an agent should stop and ask a lawyer for review. Frequent checkpoints keep lawyers in control and catch errors early, but they slow agents down and make them feel like assistants rather than autonomous helpers. Fewer checkpoints are faster and more impressive, but errors can compound and lawyers may feel uneasy about work they didn't supervise.
The prompt: "For Harvey Agents, should we add more lawyer review checkpoints or give agents more autonomy?" I'll analyse it as a product trade-off.
I'll ask clarifying questions, identify the trade-off type, analyse the product, form a hypothesis, choose metrics, design an experiment, plan analysis, set a decision framework, and recommend next steps.
Step 1
Clarifying Questions (3 minutes)
Why it matters: A research memo, a contract markup and a document review have different risks. Expected answer: Start with a contract review agent that reviews a third-party contract against a playbook and drafts a redline. Impact on approach: Errors are reviewable before anything leaves the firm, which limits risk.
Why it matters: Shows the baseline. Expected answer: Some find the current agent slow because it asks for approval after every step; others like seeing progress. Impact on approach: Preferences differ, so configuration may matter.
Why it matters: Speed or trust. Expected answer: Increase completed agent runs and lawyer satisfaction without increasing errors. Impact on approach: Balance throughput and quality.
Why it matters: Enables adaptive checkpoints. Expected answer: Yes, with reasonable calibration on clause classification. Impact on approach: Adaptive checkpoints become an option.
Practice similar questions
Subscribe to access the full answer
.png){kind=logo}
