Currently, these systems have no way to separate trusted and untrusted input. This leaves them vulnerable to prompt injection attacks in basically any scenario involving unvalidated user input. It’s not clear yet how that can be solved. Until it has been solved, it seriously limits how developers can use LLMs without opening the application up to exploitation.
Thanks for the feedback, I appreciate it!
While it’s true that prompt injection is a problem, I’m not sure that it seriously limits how we engineers can utilize LLMs.
Take for instance a banking application. In order to interact with the application via any interface, I must be authenticated. Every authentication mechanism will ensure that users can only act on their own behalf. The functions/API which I allow the user to call via the Natural Language Interface are the same that I allow the user to call through the web or mobile.
In this scenario, I can take full advantage of the LLM, and not worry much about prompt injection, save for some affordances like no internet access or access to other inputs that didn’t come from the system.
I believe prompt injection becomes a concern when you have the LLM ingest untrusted data. e.g. reading stock price from an external site could open your system up to a prompt injection.
Currently, these systems have no way to separate trusted and untrusted input. This leaves them vulnerable to prompt injection attacks in basically any scenario involving unvalidated user input. It’s not clear yet how that can be solved. Until it has been solved, it seriously limits how developers can use LLMs without opening the application up to exploitation.
Thanks for the feedback, I appreciate it!
While it’s true that prompt injection is a problem, I’m not sure that it seriously limits how we engineers can utilize LLMs.
Take for instance a banking application. In order to interact with the application via any interface, I must be authenticated. Every authentication mechanism will ensure that users can only act on their own behalf. The functions/API which I allow the user to call via the Natural Language Interface are the same that I allow the user to call through the web or mobile.
In this scenario, I can take full advantage of the LLM, and not worry much about prompt injection, save for some affordances like no internet access or access to other inputs that didn’t come from the system.
I believe prompt injection becomes a concern when you have the LLM ingest untrusted data. e.g. reading stock price from an external site could open your system up to a prompt injection.