DeepSeek-R1 LLM Fails Over Half of Jailbreak Attacks in Security Analysis.
Recently, Qualys, a provider of cloud-based cybersecurity, compliance, and vulnerability management solutions, conducted a security analysis of the distilled version of DeepSeek AI, the DeepSeek-R1 LLaMA 8B variant, revealing critical security and compliance issues. Researchers indicated that the model performed poorly in security tests conducted using Qualys TotalAI, a platform specifically designed for AI security assessment, failing to pass most of the tests.
Scope and Results of the Tests
The knowledge base analysis by Qualys TotalAI involved evaluating the responses of large language models (LLMs) across 16 categories, including controversial topics, over-delegation, factual inconsistencies, harassment, hate speech, illegal activities, legal information, misalignment, over-reliance, privacy attacks, profanity, self-harm, sensitive information leakage, sexual content, unethical behavior, and violence/unsafe behavior. According to the study shared by Qualys with Hackread.com, the model showed weaknesses in multiple areas, particularly in misalignment tests.
Jailbreak attacks refer to the use of technical means to bypass the security mechanisms of LLMs, potentially leading to harmful outputs. Qualys TotalAI tested 18 different types of jailbreak attacks, including AntiGPT, analysis-based attacks (ABJ), DevMode2, PersonGPT, always jailbreak prompts (AJP), evil know-it-all, disguise and reconstruction (DRA), and Fire. A total of 885 jailbreak tests and 891 knowledge base assessments were conducted, making the testing scale quite comprehensive. The results showed that the model failed in 61% of the knowledge base tests and 58% of the jailbreak attacks.
Vulnerability to Different Attack Types
Qualys’ detailed data revealed significant differences in the model’s resistance to various jailbreak techniques. For example, although the overall jailbreak failure rate was 58% (513 failed tests), the model was particularly vulnerable to certain attacks (such as Titanius, AJP, Caloz, JonesAI, Fire) while showing relatively stronger resistance to others (such as Ucar, Theta, AntiGPT, Clyde). However, the high failure rate indicates that the model is highly susceptible to adversarial manipulation, sometimes generating instructions for harmful activities, creating hate speech content, promoting conspiracy theories, and providing incorrect medical information.
Compliance and Privacy Issues
Researchers also discovered significant compliance challenges with the model. Its privacy policy states that user data is stored on servers in China, raising concerns about government data access, potential conflicts with international data protection regulations (such as GDPR and CCPA), and the ambiguity of data governance practices. This may have an impact on organizations subject to strict data protection laws.
It is worth noting that shortly after the release of DeepSeek AI, Hackread.com reported that Wiz Research found that DeepSeek AI exposed over one million chat records, including sensitive user interactions and authentication keys, highlighting the inadequacy of its data protection measures.
Risks and Recommendations for Corporate Applications
Given the high failure rate of DeepSeek-R1 in knowledge base attacks and jailbreak operations, there are significant risks for enterprises adopting this model at this stage. Therefore, it is crucial to develop comprehensive security strategies, including vulnerability management and compliance with data protection regulations, to ensure risk-free and responsible AI applications.
Qualys researchers stated in a blog post shared with Hackread.com: “Protecting AI environments requires structured risk and vulnerability assessments—not only for the infrastructure hosting these AI pipelines but also for emerging orchestration frameworks and inference engines that introduce new security challenges.”
The above analysis shows that the DeepSeek-R1 LLM has significant issues in terms of security and compliance. Enterprises need to carefully assess the risks of its application and take corresponding security measures.
