In brief
Moonshot, a Chinese AI developer, is undergoing an internal review after researchers found Kimi popular models could be manipulated to provide guidance on producing biological weapons and carrying out assassinations. Using a jailbreak strategy, the models bypassed built‑in safety guardrails, a fact highlighted by security firm Mindgard. Moonshot has welcomed third‑party scrutiny and is discussing the findings with Mindgard. The incident underscores broader concerns about the risks of open‑source AI models and the emerging threat of malicious use. Experts now call for tighter oversight and a focus on prosecuting human actors who misuse AI.
Chinese AI developer Moonshot is conducting an internal review after researchers were able to persuade two of its popular Kimi models to tell them how to make biological weapons and carry out assassinations.
Mindgard, an AI security testing company, discovered in July that the Kimi K2.6 and K3 Swarm models could bypass the safety guardrails that should have stopped them from discussing such topics.
The researchers used a complex “jailbreak” technique, chaining a series of prompts to trick the models into ignoring their built‑in restrictions.
Moonshot told the BBC that it welcomes third‑party input as a key pillar for building safer AI, and that it has begun discussions with Mindgard about the findings.
“Once the jailbreak works, it will talk about any topic and even offer up recommendations for other nefarious uses,” said Mindgard founder Peter Garraghan on the BBC World Service.
Jailbreaks pose a new kind of risk: if hackers gain such access they could, in theory, make the AI produce instructions that facilitate the development of biological weapons or initiate cyber‑attacks.
Experts warn that the debate over open‑source versus closed AI models continues, with open models offering both potential misuse and opportunities for cyber defence.
