Peichun Hua

arXiv preprint · 2026

Backdoor in the Loop: Compromising Agentic Search via Malicious Retrievers

Beining Xu*, Peichun Hua*, Yunming Xiao

* Equal contribution.

Summary

This paper shows that a compromised retriever checkpoint can steer an agentic search system without touching the agent or the document corpus. In agentic retrieval-augmented generation (RAG), a language model agent alternates between reasoning and retrieving passages, so the retriever shapes both the evidence the agent sees and when it stops searching. A backdoored retriever (one that misbehaves only on inputs containing a hidden trigger) can suppress supporting passages, keep returning one chosen document, or stretch the search out, and a short inject-and-unlearn step weakens the signals that backdoor detectors rely on.

Key numbers

  • 99.8–100%Share of triggered questions where the attacker's chosen document appears in every retrieval round
  • +78–88%Increase in mean search rounds at high round-control intensity, with 81–87% more context tokens and 80–126% more latency
  • 46.53 → 29.26Spectral Signatures detector separation before and after Decoy Overwrite–Unlearn, while the malicious retrieval behavior persists

Method

Retrieval-augmented generation (RAG) grounds a model in retrieved passages [1]. In the agentic setting studied here, which follows Agentic-R [2], a search agent writes a query each round, and the retriever receives the original question together with that query. The agent controls queries and the stopping decision, and the retriever controls the evidence that informs both. The threat model is a model supply-chain attack: a malicious provider ships a retriever checkpoint (a saved set of model weights) that the victim runs over its own clean corpus. The attacker needs no write access to the corpus.

The backdoor activates when a trigger word, here "author", appears in the question. Because the original question stays in every retrieval input, the trigger persists even as the agent rewrites its queries. The paper trains three behaviors. The untargeted attack suppresses supporting passages and corrupts answers. The targeted attack makes one existing document rank first in every round. The round-control attack induces longer searches at low, medium, or high intensity to inflate token and latency cost.

Concealment uses Decoy Overwrite–Unlearn (DOU). Backdoor purification methods such as BPO inject a known auxiliary backdoor and then unlearn it, hoping the removal also erases unknown backdoors. DOU runs the same cycle with a small budget so that the decoy is removed while the original backdoor survives, and the features that detectors inspect change.

Three retriever backdoor objectives—evidence suppression, persistent target retrieval, and prolonged search—followed by decoy overwrite and unlearning to conceal the backdoor.
Three agentic retrieval attacks and the Decoy Overwrite–Unlearn concealment procedure. Figure 2 in the paper

Evaluation

The experiments use HotpotQA and TriviaQA with E5 [3] and Contriever [4] retrievers over a fixed corpus and top-3 retrieval. For the targeted attack, the chosen document appears in every executed round for 99.8 to 100% of 1,000 triggered questions per setting and ranks first in more than 99.7% of rounds. Without the trigger it appears in only 1.8 to 2.7% of trajectories. Triggered F1 falls by 22.5 to 30.1 points, and searches run about one round longer. Ordinary questions that naturally contain the word "author" also activate the backdoor, with F1 down 13.9 to 34.7 points on 90 such questions.

At high intensity, round control raises mean search length by 78 to 88%, final context by 81 to 87% in tokens, and latency by 80 to 126%, while F1 falls by 24 to 29 points. Clean-input behavior changes little across intensities, so the extra cost lands on triggered queries.

Purification does not remove the backdoor. After clean fine-tuning and 30% magnitude pruning, the clean-versus-trigger F1 gap stays at 28.6 to 30.1 points on HotpotQA. BPO removes its auxiliary association (the target win rate falls from 100% to 0%) while the original backdoor remains. On a held-out set of 1,000 MuSiQue questions, DOU at a medium budget lowers the detector separation Δ, the mean distance of a detector's AUROC from chance (50), from 46.53 to 29.26 for Spectral Signatures [5], from 38.23 to 19.40 for Neural Cleanse, from 44.63 to 14.66 for Black-box TopK Instability, and from 41.22 to 23.83 for Activation Clustering [6]. Triggered-input F1 changes by at most 0.9 points.

Persistent target retrieval under the activated backdoor
SettingEvery-round target hitTarget ranked first per round
HotpotQA / E5100.00%99.87%
HotpotQA / Contriever99.80%99.74%
TriviaQA / E5100.00%99.96%
TriviaQA / Contriever99.90%99.84%
Selected triggered-input rows from Table 2. Each setting uses 1,000 paired questions, a fixed clean corpus, and top-3 retrieval. Table 2 in the paper

Safety and limitations

This work demonstrates an attack, so the safety question is what it enables. All experiments use public question-answering benchmarks and locally hosted retrievers, with no human subjects, real users, or production systems. The code is released to support evaluation of defenses, and the authors limit redistribution of ready-to-use backdoored checkpoints. Retriever supply chains are already exploitable, and the paper reports where seven detectors and a purification baseline fail so that defenders can act on it.

The evaluation covers one trigger word, two retriever families, and three benchmarks. Targeted backdoors are harder to conceal with DOU alone, and the paper tests an optional stability-equalization extension for them. BPO was designed for supervised image tasks, and an effective adaptation to retrieval is left for future work.

Citation

@article{xu2026backdoor,
  title={Backdoor in the Loop: Compromising Agentic Search via Malicious Retrievers},
  author={Beining Xu and Peichun Hua and Yunming Xiao},
  journal={CoRR},
  volume={abs/2609.37468},
  year={2026},
  url={https://arxiv.org/abs/2609.37468},
  doi={10.48550/arXiv.2609.37468},
  eprinttype={arXiv},
  eprint={2609.37468},
  primaryClass={cs.CR}
}

References

  1. Patrick Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS, 2020. arxiv.org/abs/2005.11401
  2. Wenhan Liu et al. Agentic-R: Learning to Retrieve for Agentic Search. arXiv, 2026. arxiv.org/abs/2601.11888
  3. Liang Wang et al. Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv, 2022. arxiv.org/abs/2212.03533
  4. Gautier Izacard et al. Unsupervised Dense Information Retrieval with Contrastive Learning. Transactions on Machine Learning Research, 2022. arxiv.org/abs/2112.09118
  5. Brandon Tran, Jerry Li, and Aleksander Madry. Spectral Signatures in Backdoor Attacks. NeurIPS, 2018. arxiv.org/abs/1811.00636
  6. Bryant Chen et al. Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. arXiv, 2018. arxiv.org/abs/1811.03728