Ethical Hacking News
In this groundbreaking study, researchers from Intruder investigate the potential of artificial intelligence (AI) in writing vulnerability checks. By exploring both traditional chatbots and "agentic" models, they discover that AI can indeed produce high-quality templates with minimal prompts. However, the team also highlights challenges such as false positives, truncated responses, and limitations in current outputs. Can we trust AI to write vulnerability checks? The answer lies in balancing human oversight with the capabilities of AI-powered tools.
Researchers explored the potential of AI in generating high-quality vulnerability checks. The initial approach using an LLM chatbot failed due to inconsistent output quality. A revised approach using an "agentic" model improved template quality and moved closer to human-engineered checks. Results demonstrated the AI-powered tool could generate high-quality checks with proper guidance, but still required occasional course corrections. Experiments showed the tool's capabilities in generating templates for various scenarios, such as exposed admin panels and unauthenticated users accessing data. Roadblocks included false positives, limitations in chatbot outputs (e.g., truncated curl responses), and drift from optimal solutions.
In an era where artificial intelligence (AI) is increasingly being touted as a silver bullet for various tasks, including vulnerability management, a growing concern has emerged regarding the reliability and trustworthiness of AI-generated vulnerability checks. This raises questions about the role of human oversight and expertise in ensuring the accuracy and quality of vulnerability assessments.
Recently, a research project was undertaken by Intruder's security team to explore the potential of AI in generating high-quality vulnerability checks. The goal was to determine whether AI could help bridge the gap between the speed required for rapid scanning and the need for precision in identifying vulnerabilities.
To begin their investigation, the researchers started with a simple approach: feeding prompts into an LLM chatbot to generate Nuclei templates. However, this initial attempt proved unsuccessful due to the inconsistent quality of the outputs generated by the chatbot. The results referenced non-existent features, produced invalid syntax, and relied on weak matchers and extractors.
Undeterred by these initial setbacks, the researchers then shifted their approach to utilize an "agentic" model rather than a traditional chatbot. This change allowed them to harness the capabilities of tools and search reference material, enabling the agent to learn from a curated repository of Nuclei templates and improve its output quality.
The results of this revised approach were significantly more promising. By layering on rules and incorporating clear prompting, the quality of the generated templates improved substantially, moving closer to what would be expected from human-engineered checks. Although the process still required occasional course corrections, the researchers noted that with the right guidance, the agent could produce high-quality checks that appeared to have been manually written.
As part of their exploration, the Intruder team experimented with different scenarios to assess the capabilities of their AI-powered tool. One such test involved creating a vulnerability check for an exposed admin panel on the internet, which proved particularly challenging due to its simplicity in principle but the need for human expertise to scale its production.
Another successful experiment was conducted using the "unsecured Elasticsearch" check, where the AI agent was tasked with identifying instances of unauthenticated users accessing data. The results demonstrated that despite initial challenges, such as truncation issues and reliance on imperfect matchers, the tool could still generate high-quality templates within a short period.
However, the researchers also encountered several roadblocks during their investigation. They found that even with the application of rules and clear prompting, the AI-powered agent occasionally drifted off course. For instance, an initial attempt at creating an exposed admin panel check resulted in weak matchers, highlighting the need for human oversight to prevent false positives.
Furthermore, issues arose due to limitations in the current outputs generated by the chatbot, such as truncated curl responses that could miss critical identifiers for ideal matchers. The team is currently working on addressing these challenges through new rule sets and guidelines.
In conclusion, while AI holds promise as a productivity tool in vulnerability management, it is far from being considered a replacement for human security engineers without proper supervision. By embracing the capabilities of both human expertise and AI-powered tools, we can strike a balance between speed and accuracy in identifying vulnerabilities.
As technology continues to evolve at an unprecedented pace, the importance of critically evaluating the role of AI in various applications cannot be overstated. In this case, by examining the successes and setbacks of using AI in vulnerability management, we gain valuable insights into the potential benefits and limitations of leveraging artificial intelligence for enhancing security assessments.
By acknowledging both the advantages and drawbacks of AI-powered tools like those developed by Intruder's research team, we can foster a more nuanced understanding of their applications and improve our collective efforts in addressing the rapidly evolving landscape of cybersecurity threats.
Related Information:
https://www.ethicalhackingnews.com/articles/Can-We-Trust-AI-to-Write-Vulnerability-Checks-A-Study-on-the-Use-of-Artificial-Intelligence-in-Vulnerability-Management-ehn.shtml
https://www.bleepingcomputer.com/news/security/can-we-trust-ai-to-write-vulnerability-checks-heres-what-we-found/
Published: Mon Sep 29 10:14:52 2025 by llama3.2 3B Q4_K_M
