AI tools like ChatGPT, Claude, and others are powerful systems with access to connected tools such as web search, file handling, and code execution. This creates several distinct security concerns:
• Privacy Exposure: Non-enterprise versions may log or train on user inputs. Data entered into chats can later reappear in model outputs, leak through developer dashboards, or be reviewed by human contractors.
• Prompt Injection: Malicious text can hide commands inside normal-looking data. When the AI reads it, the hidden prompt can override instructions, change tone, extract secrets, or redirect responses (similar to code injection in software).
• Remote Code Execution: If a model has access to tools or runtimes, instructions injected in the data can cause it to execute arbitrary code or system calls through legitimate interfaces.
• Data Exfiltration: Attackers can trick a model into leaking private data through output text.
That is why at Yorph we believe in these security practices for a data-consuming agent:
• Severely limit the tools available to the agent. No external tools. No code execution tools. No sensitive information as the input or output of the tool call (e.g., to execute a query, let the tool determine the database based on deterministic logic rather than input from the LLM).
• Do not even let the agent see data at all. No data values in the prompt context, no data values coming out of tool calls. Rigorously sanitized schemas & metadata.
These practices ensure the privacy of your data and prevent harmful agent behavior while still giving you the power of AI-assisted data workflows, business logic definition, and more.