Amazon announced Tuesday it is expanding its cybersecurity efforts by launching a dedicated bug bounty program for its generative artificial intelligence models, including its NOVA suite. This initiative invites select third-party security researchers and academic institutions to identify and report vulnerabilities within the company’s AI tools.
The new program will focus on common risks associated with large language models, such as prompt injection, jailbreaking, and other exploitable weaknesses. Researchers will also examine the potential for misuse of these models in developing weapons of mass destruction. Successful reporting of valid findings will result in monetary compensation. This move by Amazon comes as the company increases its investment and reliance on generative AI technologies across its diverse product and service offerings.
Amazon’s AI Bug Bounty Program Focuses on Security
The primary goal of Amazon’s new AI bug bounty program is to proactively identify and mitigate security risks before they can be exploited. By offering incentives to external researchers, the company aims to leverage a wider pool of expertise to uncover flaws that internal teams might miss.
Amazon stated that security researchers act as crucial validators for the robustness of their AI systems. This collaborative approach acknowledges the complex and evolving nature of AI security, requiring constant vigilance and innovative testing methods. The program’s structure, however, remains invite-only, meaning Amazon will curate the participants.
Scope of AI Vulnerabilities Being Tested
The program targets specific categories of vulnerabilities inherent in generative AI. These include:
- Prompt Injection: Attempts to manipulate the model’s output through carefully crafted inputs.
- Jailbreaking: Efforts to bypass safety restrictions and elicit prohibited content or actions from the AI.
- Real-world Exploitation Potential: Vulnerabilities that could be practically leveraged by malicious actors.
- WMD Development Assistance: Investigating how the AI could be prompted to provide information on creating chemical, biological, radiological, or nuclear weapons.
This comprehensive approach underscores Amazon’s commitment to responsible AI development and deployment. The company has previously engaged in similar security testing, including a tournament held earlier this year that awarded significant prizes for identifying bugs and vulnerabilities in Amazon’s coding AI models.
This AI bug bounty program for NOVA models represents a significant step in Amazon’s broader strategy to integrate generative AI while maintaining a high standard of security. The selection process for researchers is expected to begin next year.