A document can control your AI without you noticing

Imagine an ordinary working day. You upload a supplier contract to an AI assistant and ask it to identify risky clauses. A few seconds later, you receive a clear summary. However, some passages are missing, while others sound far more favourable than they really are. The problem may not be that the AI misunderstood the contract. It may simply have followed an instruction hidden inside the document. You did not see it. The AI did.

We share sensitive data with AI without thinking

AI now summarises contracts, reviews proposals, analyses spreadsheets and prepares email responses. Along with the task, it often receives personal data, internal policies, financial results, price proposals, source code, customer communications or contracts that have not yet been finalised. The user sees a tool that saves an hour of work in a few seconds. They think less about where the document is sent, how long it remains there and who may gain access to it.

How an AI provider handles inputs depends on the specific service, account type and configuration. Not every uploaded document is automatically used to train the model. However, a company should know where its data is processed, how long it is retained, who can access it and whether its use can be controlled. If the company cannot answer these questions, it does not have control over its data.

A document may no longer be just a document

Traditional software processes commands and data separately. A language model, however, works with text that may contain the user’s original request, system rules and the content of the document being analysed. It may not always recognise the boundary between data and instructions correctly.

If you ask AI to summarise a contract, it should treat the contract only as a source of information. The text may, however, contain an instruction telling it to ignore the original request, conceal a particular clause or change the final response. The instruction can be hidden in metadata, an invisible document layer, an image or text that blends into the background. A person may not notice it. To the AI, it is still content that may be interpreted as a command.

An attack that does not require your password

This type of attack is called indirect prompt injection. The attacker does not need to enter the corporate network or gain access to the AI assistant. It is enough to prepare content that the user later provides to the AI.

A malicious instruction can be placed inside a contract, résumé, proposal, email attachment, website or document stored in an internal knowledge base. With multimodal AI, it may also be embedded in an image or other content that the system can analyse. The attacker does not even need to know who will open the document or when. They only need to create content that is likely to be processed by AI.

What can a hidden instruction do?

The consequences depend on the data and permissions available to the AI. If it only summarises one document, the attacker may influence the answer, conceal an unfavourable clause or highlight information that serves their interests. If the AI can access internal documents, emails or databases, it may attempt to find and disclose additional sensitive information.

The risk becomes even greater with AI agents that can use corporate tools. A manipulated AI may do more than generate an incorrect response. It could prepare or send a message, change a system record, open a dangerous link or perform another action on behalf of the user. The more data and permissions AI receives, the more opportunities it gives an attacker.

The Open Worldwide Application Security Project, an organisation that develops methodologies and recommendations for application security, identifies prompt injection as one of the main risks associated with LLM use. A successful attack may lead to information disclosure, manipulated results and the misuse of connected systems.

Not every AI system presents the same level of risk

An isolated AI tool that processes a single document and cannot access other systems presents a lower risk. It may still generate a manipulated response or handle data inappropriately, but its capabilities are limited.

The risk increases significantly when AI can search internal documents, read emails, process customer data or perform actions. An AI agent with access to corporate systems is not merely a smarter chatbot. From a security perspective, it is another user whose permissions must be defined, whose activity must be monitored and whose potential impact must be limited.

One sentence in the system prompt is not enough

A simple defence may be to instruct the AI to ignore commands found in documents and never disclose sensitive information. Such a rule may help, but it is not a reliable security boundary. An attacker can change the wording, language, structure or position of the instruction and test when the model begins to prioritise it.

There is currently no universal protection capable of stopping every prompt injection attack. Organisations must therefore consider the possibility that a malicious instruction may bypass protective controls. The important step is to limit both the data the AI can access and the actions it can perform.

How to reduce the risk

AI should not automatically receive the same permissions as the user. If it only needs to analyse documents, it does not need permission to modify, delete or send them. If it prepares an email draft, it should not send it without the user’s confirmation. Sensitive actions must remain under human control.

An external document should not be considered safe simply because it contains no virus. For AI, even an ordinary sentence may be dangerous. Organisations therefore need to monitor what data the AI processes, which tools it accesses and whether it attempts to perform an action unrelated to the original request.

Regular security testing is equally important. Functional testing verifies whether AI can summarise a contract correctly. Security testing determines whether it can be manipulated into hiding information, disclosing sensitive data or misusing a connected system.

Banning AI will not solve the problem

If a company bans AI but employees find it useful, some of them will continue using it through personal accounts and without the security team’s knowledge. This creates shadow AI, which means the use of unapproved AI tools outside the organisation’s control.

A better approach is to provide approved tools and define clearly which data may be processed, which data must be anonymised and which information must never enter an external system. A general rule against entering sensitive data is not enough. Employees need to know whether it also applies to contracts, résumés, proposals and meeting notes.

Who is really controlling your AI?

AI may be controlled by the user who assigned the task. It may be guided by rules defined by a developer. In an inadequately secured application, however, its behaviour may also be influenced by the author of the document it is processing.

Organisations do not need to stop using AI. They do, however, need to know what data they provide, what content the system is allowed to load and what actions AI may perform on behalf of a user.

A document may no longer contain only information. It may also contain a command.

Can your AI resist hidden instructions?

Citadelo tests the security of AI and LLM applications, including their resistance to prompt injection attacks, sensitive data disclosure and the misuse of connected tools.