C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation

Guoxin Chen1, Minpeng Liao1,†, Peiying Yu1, Dingming Wang2,
Zile Qiao1, Chao Yang3 Xin Zhao4,† Kai Fan1,†
1Tongyi Lab,
2University of Oxford, 3Tsinghua University 4Renmin University of China
†Corresponding author
Paper arXiv Code Demo
Evaluation of 12 Large Multi-Modal Models

Overall framework of C-3PO. (Upper left) Essential cognitive capabilities required for effective RAG system interaction in human-guided alignment. (Upper right) Our proxy-centric alignment simulates these human-like interaction through a lightweight multi-agent system with collaborative strategies. (Bottom) The end-to-end optimization pipeline for our multi-agent system.

Abstract

Retrieval-augmented generation (RAG) systems face a fundamental challenge in aligning independently developed retrievers and large language models (LLMs). Existing approaches typically involve modifying either component or introducing simple intermediate modules, resulting in practical limitations and sub-optimal performance. Inspired by human search behavior—typically involving a back-and-forth process of proposing search queries and reviewing documents, we propose C-3PO, a proxy-centric framework that facilitates communication between retrievers and LLMs through a lightweight multi-agent system. Our framework implements three specialized agents that collaboratively optimize the entire RAG pipeline without altering the retriever and LLMs. These agents work together to assess the need for retrieval, generate effective queries, and select information suitable for the LLMs. To enable effective multi-agent coordination, we develop a tree-structured rollout approach for reward credit assignment in reinforcement learning. Extensive experiments in both in-domain and out-of-distribution scenarios demonstrate that C-3PO significantly enhances RAG performance while maintaining plug-and-play flexibility and superior generalization capabilities.

Experiment Results

In-domain Results

Experiment Main Results

Out-of-generalization Results

Out-of-generalization Results

Reference

@misc{chen2025c3pocompactplugandplayproxy,
      title={C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation}, 
      author={Guoxin Chen and Minpeng Liao and Peiying Yu and Dingmin Wang and Zile Qiao and Chao Yang and Xin Zhao and Kai Fan},
      year={2025},
      eprint={2502.06205},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.06205}, 
    }