In our recent paper on prompt injections, we derived new threats facing applications built on top of LLMs. In this post, I will take these abstract threat models and show how they will affect software being deployed to hundreds of millions of users- including nation-states and militaries. We will look at LLM applications in escalating order of stupidity, ending with attackers potentially compromising military LLMs to suggest kinetic options (a euphemism for bombing people with battlefield AIs compromised by bad actors)⊕ I guess it would be an evil use-case no matter if the battlefield AI was compromised by other evil actors- icing on the cake.
This is a very interesting read given the hard press on AI that my company is pushing.
I guess I’ll try to make sure that we don’t implement some of these really bad ideas.
A lot of these seem to go away if you don’t connect to the Internet or allow user input, at least.