The Rise of Autonomous Hackbots in Cybersecurity

I’ve met with, seen, and used a bunch of the top hackbots being built by different startups. While there is some variation in ability, they’re all impressive in different ways. Watching a bunch of agents spin-off to hack different components is both exciting and terrifying.

What Exactly Is a Hackbot?

In simple terms, a hackbot is an AI-driven tool designed to autonomously identify and exploit vulnerabilities. Unlike traditional automated scanners, hackbots leverage advanced machine learning techniques, such as large language models (LLMs) and reinforcement learning, to dynamically and intelligently hack applications. Scanners are much more rigid and cannot adapt to any application the way that hackbots can due to their use of AI.

The Current Landscape: Building Blocks and Real-World Applications

Three main building blocks make hackbots possible:

• Large Language Models (LLMs): Give hackbots the ability to read, write, and interact in human-like language, allowing them to adapt to any application on the fly.
• Tool Integration: Connect hackbots to browsers, fuzzers, and scanners to automate complex tasks. But what makes them work are “harnesses” that let them interact with apps in the browser and tweak HTTP requests and responses as needed.
• Computer-Use Ability: Computer use is still a form of a “tool,” but it’s unique and important enough to warrant its own section. Computer use allows LLMs to use a computer (or at least a browser) like a human. It’s key for hackbots to be able to achieve human-like hacking abilities because it’s required to understand all the features of modern applications.

The hackbot landscape is evolving rapidly with several distinct trends emerging. There are some real-world examples of hackbots and hackbot-like applications already:

• The first and most obvious is that people are using AI to improve manual hacking by using LLMs to write scripts, build automation, write exploits (such as XSS WAF bypasses and CSRF POCs), and write reports
• Early-stage startups and mature companies alike are developing, testing, and deploying full blown hackbots which can be pointed at websites and autonomously find vulnerabilities through a black box approach.
• There are paid AI Security applications like Burp AI and Shift, which make security testing faster and easier by using AI to supercharge the usage of proxy software like Burp and Caido.
• There are open-source projects like cewlai and ffufai (and probably others), which use AI to enhance singular parts of the recon process while hacking—subdomain enumeration and content discovery, respectively. 

Why This Matters: Opportunities and Threats

The implications of hackbots are huge, presenting both significant opportunities and serious threats. On the positive side, hackbots have the potential to revolutionize the security of the planet by pentesting every website. Basically, by autonomously and intelligently probing for vulnerabilities, hackbots could make security assessments vastly more comprehensive and accessible, effectively lowering the cost of security testing for organizations or sectors that traditionally struggle with limited resources.

That said, the potential misuse of hackbots is equally existent. Once they’re built, these autonomous hacking tools can amplify the leverage of bad actors exponentially, surpassing the scale and speed achievable by human attackers alone. More than that, the autonomous nature of hackbots introduces complexities in attribution, making it harder to determine responsibility and trace attacks back to their origin.

Additionally, hackbots face technical and ethical hurdles. Accuracy remains a tough challenge, as these tools have to handle false positives and negatives through sophisticated, context-aware analysis. They could definitely break things in their testing.

And there's also the inherent risk of the hackbots themselves becoming targets, vulnerable to threats like prompt injection attacks, which could compromise their integrity and turn defensive tools into liabilities.

Ethically, the development and dissemination of powerful offensive AI capabilities pose difficult questions as well. Careful consideration will need to be taken when considering limits and oversight required for responsible use. From a legal standpoint, hackbots raise questions about compliance with regulations like the Computer Fraud and Abuse Act (CFAA) or international cybersecurity laws.

The Future of Hackbots

I believe that Hackbots will reach maturity right around the same time that automated code-review reaches decent quality. Between now and then, the amount of total code deployed will go up significantly as the barrier to entry to developing and deploying code is going down. The new “vibe coding” trend means that millions of people who aren’t developers can now create applications opening the door for more errors and vulnerabilities. 

This increase in risk, combined with an increase in security tooling, makes it very difficult to anticipate how the overall security landscape will change. A lot more bugs will be introduced, but there will be really amazing ways to increase security (hack bots and AI code review). 

It seems clear to me that human oversight will continue to be essential in the near term, but the role of humans will shift dramatically in the next ten years. The cybersecurity and AI communities are not currently integrated. Right now, the AI labs are worried about hacking ability in their models, and the hacking security community is perfect to asses those abilities.  It would be much better if they came together to discuss and work on this tough problem together.  I believe security is the frontier of AI model improvement, and I can’t wait to see how everything shakes out.