Key Takeaways
- • Aikido's AI pentesting platform identified 49 verified vulnerabilities across two test applications versus XBOW's 31—a 58% performance differential at identical pricing tiers, according to an independent Doyensec benchmark conducted May 27, 2026.
- • False positive rates were nearly equivalent (Aikido 4%, XBOW 3%), indicating the coverage gap reflects genuine detection capability rather than noise or precision trade-offs between the platforms.
- • Vulnerability overlap between the two tools was minimal: only 3 matching findings on the Fider application and 4 on Photoview, meaning the platforms identified almost entirely different vulnerabilities despite scanning identical code—fewer than 10% agreement across 80 total findings.
- • Aikido demonstrated superior operational efficiency with sub-20-minute setup, same-day reporting, unlimited free retests, and zero scan interruptions, while XBOW required several days for setup, 5-day reporting lag, single retests within 30 days, and experienced multiple crashes during testing.
- • High/Critical severity findings favored Aikido at 9 versus XBOW's 5, with Aikido also surfacing 32 Low/Medium vulnerabilities compared to XBOW's 18.
Summary
Doyensec, an independent application security consultancy, conducted a controlled head-to-head evaluation of two AI-powered penetration testing platforms—Aikido and XBOW—on May 27, 2026. The benchmark tested two real applications randomly selected from a pool of 442, employed identical pricing tiers and credentials, and subjected all findings to manual validation by two researchers with peer review, establishing a rigorous methodology for comparative assessment.
The quantitative results showed a substantial performance gap. Aikido identified 49 verified vulnerabilities across the test applications (Fider, an open-source user feedback platform, and Photoview, a TypeScript/Next.js photo gallery application with role-based access control), while XBOW identified 31—representing a 58% advantage for Aikido at equivalent cost. Critically, this differential cannot be attributed to detection methodology differences: both platforms maintained similar false positive rates (Aikido 4%, XBOW 3%), indicating comparable precision calibration.
The most revealing metric was vulnerability overlap: across the two applications, the platforms achieved fewer than 10% agreement, with only 3 matching findings on Fider and 4 on Photoview out of 49 and 31 total findings respectively. This near-complete divergence suggests the two tools employ fundamentally different detection approaches, each identifying distinct vulnerability classes the other misses—a finding with significant implications for practitioners relying on single-platform assessments.
Operational performance also favored Aikido substantially. The platform required less than 20 minutes for setup versus XBOW's multi-day requirement, delivered reporting on the same day compared to XBOW's 5-day lag, and offered unlimited free retests rather than a single retest within 30 days. Additionally, Aikido experienced zero scan interruptions while XBOW encountered multiple crashes and restarts during testing. Severity-weighted analysis showed Aikido surfacing 9 High/Critical vulnerabilities versus XBOW's 5, alongside 32 Low/Medium findings compared to XBOW's 18.
For institutional investors evaluating AI-driven application security vendors, this benchmark underscores the material performance variance across competing platforms and the risks of vendor consolidation. The limited overlap between detection profiles suggests portfolio diversification of security tools may remain operationally necessary despite platform maturation claims in the AI pentesting sector.
The quantitative results showed a substantial performance gap. Aikido identified 49 verified vulnerabilities across the test applications (Fider, an open-source user feedback platform, and Photoview, a TypeScript/Next.js photo gallery application with role-based access control), while XBOW identified 31—representing a 58% advantage for Aikido at equivalent cost. Critically, this differential cannot be attributed to detection methodology differences: both platforms maintained similar false positive rates (Aikido 4%, XBOW 3%), indicating comparable precision calibration.
The most revealing metric was vulnerability overlap: across the two applications, the platforms achieved fewer than 10% agreement, with only 3 matching findings on Fider and 4 on Photoview out of 49 and 31 total findings respectively. This near-complete divergence suggests the two tools employ fundamentally different detection approaches, each identifying distinct vulnerability classes the other misses—a finding with significant implications for practitioners relying on single-platform assessments.
Operational performance also favored Aikido substantially. The platform required less than 20 minutes for setup versus XBOW's multi-day requirement, delivered reporting on the same day compared to XBOW's 5-day lag, and offered unlimited free retests rather than a single retest within 30 days. Additionally, Aikido experienced zero scan interruptions while XBOW encountered multiple crashes and restarts during testing. Severity-weighted analysis showed Aikido surfacing 9 High/Critical vulnerabilities versus XBOW's 5, alongside 32 Low/Medium findings compared to XBOW's 18.
For institutional investors evaluating AI-driven application security vendors, this benchmark underscores the material performance variance across competing platforms and the risks of vendor consolidation. The limited overlap between detection profiles suggests portfolio diversification of security tools may remain operationally necessary despite platform maturation claims in the AI pentesting sector.
Go deeper than the headline
Research every firm in this story, track the ones that matter and get the daily briefing — free for 7 days.