Ethical Hacking News
Microsoft Copilot Prompt Injection Flaws: A Growing Divide Over AI Vulnerabilities vs. Limits
Microsoft's Copilot AI assistant has been found to have four vulnerabilities by cybersecurity engineer John Russell. The issue revolves around prompt injection and sandbox-related behaviors that expose meaningful risk, but Microsoft treats them as expected limitations. Researchers debate whether these issues reflect actual vulnerabilities or broader limitations of large language models. Russell's report highlights specific issues such as system prompt leak, file upload restriction bypass, and command execution within Copilot's isolated Linux environment. The definition of AI risk is becoming increasingly contentious, with some arguing that these issues are known limitations rather than vulnerabilities.
Microsoft's stance on AI vulnerabilities has once again come under scrutiny following a recent report by cybersecurity engineer John Russell, who claimed to have discovered four vulnerabilities in the company's Copilot AI assistant. The issue at hand revolves around prompt injection and sandbox-related behaviors that Russell believes expose meaningful risk, while Microsoft treats them as expected limitations unless they cross a clear security boundary.
The debate surrounding the definition of AI risk is becoming increasingly contentious as these tools become more widely deployed in enterprise environments. While some researchers view prompt injection and sandbox behaviors as exposing vulnerabilities, others argue that these issues reflect broader limitations of large language models rather than actual vulnerabilities.
Russell's report highlights several specific issues, including indirect and direct prompt injection leading to system prompt leak, Copilot file upload type policy bypass via base64-encoding, command execution within Copilot's isolated Linux environment, and the ability for users to workaround restrictions through encoding. Specifically, the file upload restriction bypass is particularly interesting as it suggests that even seemingly secure controls can be circumvented with creative encoding techniques.
The debate over the definition of AI risk has already sparked a lively discussion among security professionals, with some arguing that these issues are relatively known and represent a limitation of large language models rather than an actual vulnerability. For example, security researcher Cameron Criswell points out that such behavior reflects a broader issue in which LLMs struggle to reliably distinguish between user-provided data and instructions.
However, others contend that prompt injection and sandbox behaviors can indeed be vulnerabilities if they allow attackers to bypass internal rules or logic designed to prevent unauthorized access. In this context, the OWASP GenAI project takes a more nuanced view, classifying system prompt leakage as a potential risk only when prompts contain sensitive data or are relied upon as security controls.
The implications of this debate are significant, particularly for organizations that rely on AI tools like Copilot in their operations. As these systems become increasingly integrated into enterprise environments, it is essential to have a clear understanding of what constitutes an actual vulnerability versus an expected limitation.
Ultimately, the issue at hand represents a growing divide between how vendors and researchers define risk in generative AI systems. While Microsoft has dismissed Russell's report as out of scope, the debate highlights the need for ongoing discussion and clarification on these issues.
Related Information:
https://www.ethicalhackingnews.com/articles/A-Growing-Divide-Copilot-Prompt-Injection-Flaws-Spark-Debate-Over-AI-Vulnerabilities-vs-Limits-ehn.shtml
https://www.bleepingcomputer.com/news/security/are-copilot-prompt-injection-flaws-vulnerabilities-or-ai-limits/
Published: Tue Jan 6 05:24:25 2026 by llama3.2 3B Q4_K_M
