What factors are contributing to the increased false positive rate in Qualys's Web Application Scanning (WAS) module over the last month?

Introduction

The increased false positive rate in Qualys's Web Application Scanning (WAS) module over the last month is a critical issue that demands immediate attention. This problem not only affects the product's reliability but also impacts user trust and potentially the company's market position. I'll approach this analysis systematically, focusing on identifying the root cause, validating hypotheses, and developing both short-term fixes and long-term solutions.

Step 1

Clarifying Questions (3 minutes)

• Looking at the timing, I'm thinking there might have been a recent update. Has there been any significant change to the WAS module or its underlying algorithms in the past 1-2 months?

Why it matters: Recent changes often correlate with performance shifts. Expected answer: Yes, there was an update to improve detection rates. Impact on approach: If confirmed, we'd focus on the update's impact and potential rollback strategies.

• Considering user segments, I'm curious about the distribution. Is the increased false positive rate uniform across all customers, or are certain types of applications or industries more affected?

Why it matters: Helps identify if the issue is global or specific to certain use cases. Expected answer: The issue is more prevalent in e-commerce applications. Impact on approach: We'd investigate e-commerce-specific factors and potentially segment our solution.

• Thinking about external factors, have there been any significant changes in web application architectures or technologies that our scanning might not be adapting to well?

Why it matters: External tech shifts can impact scanning accuracy. Expected answer: There's been an increase in serverless and microservices architectures. Impact on approach: We'd need to assess our scanning capabilities for these modern architectures.

• Regarding performance metrics, has there been any change in how we're measuring or defining false positives recently?

Why it matters: Ensures we're comparing apples to apples in our analysis. Expected answer: No changes in measurement methodology. Impact on approach: Confirms the issue is with the product, not the metrics.