Why Periodic Pentesting Falls Short: The Case for Continuous Security Validation

Jay Bălan has been in cybersecurity since the days of Back Orifice and early Red Hat exploits. Today he's the CISO at Super Technologies, a fast-scaling global company with thousands of repos, hundreds of thousands of cloud resources, and roughly 1,000 developers. 

When he joined Super Technologies, he launched a second bug bounty program. Jay described his approach during a recent conversation with HackerOne co-founder and CTO Alex Rice.

"A single critical vulnerability can put a hamstring on the entire business," Jay said during a recent conversation with HackerOne co-founder and CTO Alex Rice. "So identifying the critical areas was one of the first priorities."

The philosophy of continuous security validation over periodic assessment proved right. It also maps directly to a problem most enterprise security leaders are grappling with right now: the typical testing cadence no longer matches the speed at which attack surfaces change.

The Real Problem With Periodic Testing

Most enterprises still run security programs built around a compliance calendar, with quarterly pen tests, annual assessments, and the occasional red team exercise. It looks structured and checks the boxes.

"Periodic pentests, which is the current norm, just feel like a symbolic task rather than an actual measure," Jay said.

Continuous security validation starts where periodic testing stops.

In a time when the vast majority of organizations are expanding their AI footprints, we see that vulnerabilities are growing simultaneously. The 9th Annual Hacker-Powered Security Report found that prompt injection reports jumped 540% in 2025, and valid AI-related vulnerability reports grew 210% year-over-year¹.

Recent AI Security research shows that the attack surface is changing shape faster than a quarterly cycle can track, and organizations with the lowest AI testing coverage face $730K more in annual remediation costs than those who test comprehensively².

When your infrastructure is in constant motion, with new acquisitions, new releases, and new AI integrations, a test from three months ago tells you almost nothing about your posture today.

When AI Floods the Pipeline

The challenge compounds as AI-assisted development accelerates. Jay described what happened at Super Technologies when HackerOne's agentic AI testing capability, Mythos, started rolling out: reports came in at 6x, possibly 10x, prior volume, almost all AI-generated.

"We initially saw it as AI slop, and we were annoyed," he said. "But we did also find, I think, about three criticals this year."

Three criticals that a purely manual, periodic program likely would have missed. Alex Rice framed the dynamic: AI tools need specific harnesses and orchestration and instruction on where you want to point them.

"We still get extremely cool vulnerabilities reported through HackerOne," Jay said, "and when we look at them, this could not be found by AI. AI probably will not have an easy time doing SSRFs. I'm not entirely sure AI can do dependency confusion."

In the Hacker-Powered Security Report, researchers identified business logic flaws as the vulnerability class AI tools are weakest at finding with multi-step exploits and authentication bypasses close behind. These are exactly the classes that represent real exploitable risk and require human creativity.

What Continuous Security Validation Actually Looks Like

Jay's approach at Super Technologies is a useful model. He divides the work deliberately: the internal red team handles white-box pen testing, new builds, and red team agents for vulnerability management. The external surface goes to the bug bounty community.

"Bug bounty cannot do white box pen testing. They don't have access to the source code, they don't see your infrastructure like we see it," he explained. "So the red team will have first-hand experience with new builds, new releases, new products. And we can leave the external surface with the bug bounty."

That division of labor creates continuous coverage without burning out internal resources, and it ensures an outside perspective on the surface that's actually exposed to attackers.

Getting to Proof That Matters

When a bug bounty researcher finds something, it changes the internal conversation. 

"Every now and then, when a vulnerability is identified and discovered, and then you see the impact of it, you can use that," Jay said. 

Real findings accelerate architectural changes already on the roadmap and give security teams evidence to prioritize against competing demands.

"Look, somebody else already found it. Maybe we should put more priority on this project," he said.

A validated, exploitable vulnerability with demonstrated business impact carries a different weight than a theoretical scanner finding or a compliance checkbox. It moves security decisions up the calendar and off the backlog.

Making Continuous Security Stick

Jay also shared that his biggest mistake was keeping too much centralized in the security team.

"I believe that we know how to handle security better than anybody in the organization," he said. "But one of the mistakes I was making was taking all of that exclusively on us."

The fix was embedding security into engineering culture through a security champions program, working with engineers rather than around them, and using real vulnerabilities as teaching moments. Red team findings became "magic tricks" that made engineers want to understand how the attacks worked.

That cultural shift to continuous validation only works if the organization can close the loop, remediating quickly enough to reduce exposure before the next change. Slow internal processes become the bottleneck regardless of how good the testing is. 

The full model of continuous security validation, covering discovery, validated risk, prioritization, and fast remediation, is what separates programs that reduce annual impact from those that just document it.

What Continuous Security Validation Requires

Jay's framework translates cleanly to a maturity check for any enterprise security leader:

• Can you demonstrate formal testing coverage of 91% or more of your AI and critical systems?
• Do you have continuous testing on your external surface, not just periodic assessments?
• Are you using layered methods (internal red team, external community, automated testing) that each find different classes of issues?
• When a critical finding surfaces, can you use it to accelerate remediation and organizational change?
• Are you tracking application logic vulnerabilities, the ones no CVE database will ever capture?

If any of those are uncertain, the gap is costing you. The question is whether you find out from your own program, or from an attacker.

Test at the speed of your attack surface with H1 Continuous Testing

^{1. Hacker-Powered Security Report 2025: The Rise of the Bionic Hacker}

^{Survey methodology: HackerOne and UserEvidence surveyed 99 HackerOne customer representatives between June and August 2025. Respondents represented organizations across industries and maturity levels, including 6% from Fortune 500 companies, 43% from large enterprises, and 31% in executive or senior management roles. In parallel, HackerOne conducted a researcher survey of 1,825 active HackerOne researchers, fielded between July and August 2025. Findings were supplemented with HackerOne platform data from July 1, 2024 to June 30, 2025, covering all active customer programs. Payload analysis: HackerOne also analyzed over 45,000 payload signatures from 23,579 redacted vulnerability reports submitted during the same period.}

^{2. Closing the AI Security Gap: Containing Risk Before It Scales}

^{Survey methodology: HackerOne surveyed 303 security leaders between January and February 2026. Respondents were screened to ensure they oversee or contribute to tracking, managing, or testing their organization’s AI/ML systems, and represent a range of senior security and offensive security roles within organizations reporting $250 million or more in revenue across the United States, Canada, the United Kingdom, Australia, Singapore, and Germany. Respondents represented multiple industries, led by Technology Hardware/Software (37%) and Banking/Financial Services/Insurance (16%), with additional representation across manufacturing, healthcare, retail/e-commerce, and other sectors.}