Data Poisoning
This is a member-only chapter. Log in with your Signal Over Noise membership email to continue.
Log in to readModule 2 · Section 6 of 7
Data Poisoning
Data poisoning is an attack on AI systems themselves rather than on users of AI. It involves introducing corrupted or misleading data into the training data or knowledge base that an AI system uses.
The practical threat for most readers is not an attacker poisoning a major model — that requires access. The more relevant version is poisoning the AI systems your organisation builds or configures: a custom GPT trained on internal documents, a retrieval-augmented generation system that pulls from a shared knowledge base, a chatbot that answers questions using a document repository.
If an attacker can introduce documents into that repository — through a compromised account, a shared folder, or social engineering — they can influence what the AI says. Imagine a company chatbot that advises employees on HR policies, poisoned with documents that give incorrect guidance about expense approvals or data handling procedures.
What to watch for: Any internal AI system that relies on user-contributed or externally sourced content for its answers. These systems need access controls on what can be added to the knowledge base, and periodic auditing of whether the outputs make sense.