Artificial intelligence came up with 40,000 new biological weapons in just six hours

Employees of the private pharmaceutical company Collaborations Pharmaceuticals conducted an experiment on the potential for abuse when using artificial intelligence. The result was shocking – it took just a couple of commands for the AI to start making dangerous biological formulas instead of creating drugs. Some of them turned out to be far more lethal than real biological and chemical weapons.

Details are extremely scarce, the participants in the experiment classified everything and generally regret that they decided to tell about their initiative. On the other hand, they uncovered a dangerous vulnerability that sooner or later someone would have stumbled upon. And it is better to think about measures to prevent such situations when working with AI now, the researchers urge.

They used a generative neural network MegaSyn, which is trained to identify toxic molecules and their combinations that pose a risk to proteins in the human body. The researchers simply replaced the target in the problem statement with the opposite, and instead of rejecting dangerous compounds, the AI began to conserve them. In just 6 hours of work, it created an impressive collection of 40,000 hypothetical poisons, and when it was tasked to select an analogue of the nerve gas VX, the AI effortlessly coped with the task.

None of what the AI came up with was created in reality, of course. However, it was not that difficult with an equipped laboratory. The experiment showed how easy it is to use useful technology for negative purposes – without having to be a true villain. The AI simply did its job without thinking about the consequences, which in reality could be catastrophic.