RAG for Text

• Question Answering

  Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

  REALM: Retrieval-Augmented Language Model Pre-Training

  Knowledge Graph Based Synthetic Corpus Generation for Knowledge-Enhanced Language Model Pre-training

  Atlas: Few-shot Learning with Retrieval Augmented Language Models

  Improving Language Models by Retrieving from Trillions of Tokens

  Self-Knowledge Guided Retrieval Augmentation for Large Language Models

  Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering

  Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

  Nonparametric Masked Language Modeling

  CL-ReLKT: Cross-lingual Language Knowledge Transfer for Multilingual Retrieval Question Answering

  One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval

  Entities as Experts: Sparse Memory Access with Entity Supervision

  When to Read Documents or QA History: On Unified and Selective Open-domain QA

  Enhancing LLM Intelligence with ARM-RAG: Auxiliary Rationale Memory for Retrieval Augmented Generation

  DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Service

• Fact verification

  CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval

• Commonsense Reasoning

  KG-BART: Knowledge Graph-Augmented {BART} for Generative Commonsense Reasoning

  What Evidence Do Language Models Find Convincing?

  Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models

• Human-Machine Conversation

  Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs

  Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory

  Internet-Augmented Dialogue Generation

  BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage

  A Model of Cross-Lingual Knowledge-Grounded Response Generation for Open-Domain Dialogue Systems

  From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL

  Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages

  Citation-Enhanced Generation for LLM-based Chatbot

  KAUCUS: Knowledge Augmented User Simulators for Training Language Model Assistants

• Neural Machine Translation

  Neural Machine Translation with Monolingual Translation Memory

  Nearest Neighbor Machine Translation

  Training Language Models with Memory Augmentation

• Event Extraction

  Retrieval-Augmented Generative Question Answering for Event Argument Extraction

• Summarization

  Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training

  Unlimiformer: Long-Range Transformers with Unlimited Length Input

  Retrieval-based Full-length Wikipedia Generation for Emergent Events

  RIGHT: Retrieval-augmented Generation for Mainstream Hashtag Recommendation

RAG for Code

• Code Generation

  Retrieval-Based Neural Code Generation

  Retrieval Augmented Code Generation and Summarization

  When Language Model Meets Private Library

  Language Models of Code are Few-Shot Commonsense Learners

  DocPrompting: Generating Code by Retrieving the Docs

  CodeT5+: Open Code Large Language Models for Code Understanding and Generation

  AceCoder: Utilizing Existing Code to Enhance Code Generation

  Syntax-Aware Retrieval Augmented Code Generation

  A^3-CodGen: A Repository-Level Code Generation Framework for Code Reuse with Local-Aware, Global-Aware, and Third-Party-Library-Aware

  SkCoder: A Sketch-based Approach for Automatic Code Generation

  CodeGen4Libs: A Two-Stage Approach for Library-Oriented Code Generation

  ToolCoder: Teach Code Generation Models to use API search tools

  CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges

  RRGcode: Deep hierarchical search-based code generation

  Code Search Is All You Need? Improving Code Suggestions with Code Search

  ARKS: Active Retrieval in Knowledge Soup for Code Generation

• Code Summary

  Retrieval-based neural source code summarization

  Retrieve and Refine: Exemplar-based Neural Comment Generation

  EditSum: A Retrieve-and-Edit Framework for Source Code Summarization

  Retrieval-Augmented Generation for Code Summarization via Hybrid GNN

  Context-aware Retrieval-based Deep Commit Message Generation

  RACE: Retrieval-augmented Commit Message Generation

  BashExplainer: Retrieval-Augmented Bash Code Comment Generation based on Fine-tuned CodeBERT

  Retrieval-Based Transformer Pseudocode Generation

  A Simple Retrieval-based Method for Code Comment Generation

  READSUM: Retrieval-Augmented Adaptive Transformer for Source Code Summarization

  Tram: A Token-level Retrieval-augmented Mechanism for Source Code Summarization

  Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)

  Cross-Modal Retrieval-Enhanced Code Summarization based on Joint Learning for Retrieval and Generation

  Automatic Smart Contract Comment Generation via Large Language Models and In-Context Learning

  UniLog: Automatic Logging via LLM and In-Context Learning

• Code Completion

  A Retrieve-and-Edit Framework for Predicting Structured Outputs

  Generating Code with the Help of Retrieved Template Functions and Stack Overflow Answers

  ReACC: A Retrieval-Augmented Code Completion Framework

  Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases

  RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation

  CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context

  RepoFusion: Training Code Models to Understand Your Repository

  Revisiting and Improving Retrieval-Augmented Deep Assertion Generation

  De-Hallucinator: Iterative Grounding for LLM-Based Code Completion

  REPOFUSE: Repository-Level Code Completion with Fused Dual Context

• Automatic Program Repair

  Repair Is Nearly Generation: Multilingual Program Repair with LLMs

  Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning

  InferFix: End-to-End Program Repair with LLMs

  RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair

  Automated Code Editing with Search-Generate-Modify

  RTLFixer: Automatically Fixing RTL Syntax Errors with Large Language Models

• Text-to-SQL and Code-based Semantic Parsing

  XRICL: Cross-lingual Retrieval-Augmented In-Context Learning for Cross-lingual Text-to-SQL Semantic Parsing

  Synchromesh: Reliable Code Generation from Pre-trained Language Models

  Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing

  RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL

  Leveraging Code to Improve In-context Learning for Semantic Parsing

  ReFSQL: A Retrieval-Augmentation Framework for Text-to-SQL Generation

  Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies

  Selective Demonstrations for Cross-domain Text-to-SQL

  Multi-Hop Table Retrieval for Open-Domain Text-to-SQL

  CodeS: Towards Building Open-source Language Models for Text-to-SQL

• Others

  De-fine: Decomposing and Refining Visual Programs with Auto-Feedback

  Leveraging training data in few-shot prompting for numerical reasoning

  Retrieval-Augmented Code Generation for Universal Information Extraction

  E&V: Prompting Large Language Models to Perform Static Analysis by Pseudo-code Execution and Verification

  Lessons from Building StackSpot AI: A Contextualized AI Coding Assistant

  Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model

RAG for Audio

• Audio Generation

  Retrieval-Augmented Text-to-Audio Generation

  Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation

  Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models

• Audio Captioning

  RECAP: Retrieval-Augmented Audio Captioning

  Audio Captioning using Pre-Trained Large-Scale Language Model Guided by Audio-based Similar Caption Retrieval

  Large-Scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation

  CNN architectures for large-scale audio classification

  Natural language supervision for general-purpose audio representations

  Weakly-supervised Automated Audio Captioning via text only training

  Training Audio Captioning Models without Audio

RAG for Image

• Image Generation

  Retrievegan: Image synthesis via differentiable patch retrieval

  Instance-conditioned gan

  Memory-driven text-to-image generation

  Re-imagen: Retrieval-augmented text-to-image generator

  KNN-Diffusion: Image Generation via Large-Scale Retrieval

  Retrieval-Augmented Diffusion Models

  Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models

  X&Fuse: Fusing Visual Information in Text-to-Image Generation

  Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs

• Image Captioning

  Memory-augmented image captioning

  Retrieval-enhanced adversarial training with dynamic memory-augmented attention for image paragraph captioning

  Retrieval-Augmented Transformer for Image Captioning

  Retrieval-augmented image captioning

  Reveal: Retrieval-augmented visual-language pre-training with multi-source multimodal knowledge memory

  SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation

  Cross-Modal Retrieval and Semantic Refinement for Remote Sensing Image Captioning

• Others

  An empirical study of gpt-3 for few-shot knowledge-based vqa

  Retrieval augmented visual question answering with outside knowledge

  Augmenting transformers with KNN-based composite memory for dialog

  Maria: A visual experience powered conversational agent

  Neural machine translation with phrase-level universal visual representations

RAG for Video

• Video Captioning

  Incorporating Background Knowledge into Video Description Generation

  Retrieval Augmented Convolutional Encoder-decoder Networks for Video Captioning

  Concept-Aware Video Captioning: Describing Videos With Effective Prior Information

  Retrieval-Augmented Egocentric Video Captioning

• Video QA&Dialogue

  Memory augmented deep recurrent neural network for video question answering

  Retrieving-to-answer: Zero-shot video question answering with frozen large language models

  Tvqa+: Spatio-temporal grounding for video question answering

  Vgnmn: Video-grounded neural module networks for video-grounded dialogue systems

• Others

  Language models with image descriptors are strong few-shot video-language learners

  RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model

  Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation

  Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval

RAG for 3D

• Text-to-3D

  ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model

  AMD: Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion

  Retrieval-Augmented Score Distillation for Text-to-3D Generation

RAG for Knowledge

• Knowledge Base Question Answering

  ReTraCk: A Flexible and Efficient Framework for Knowledge Base Question Answering

  Unseen Entity Handling in Complex Question Answering over Knowledge Base via Language Generation

  Case-based Reasoning for Natural Language Queries over Knowledge Bases

  Logical Form Generation via Multi-task Learning for Complex Question Answering over Knowledge Bases

  Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database

  RNG-KBQA: Generation Augmented Iterative Ranking for Knowledge Base Question Answering

  TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base

  DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases

  End-to-end Case-Based Reasoning for Commonsense Knowledge Base Completion

  Bridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA

  Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answering

  Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning

  FC-KBQA: A Fine-to-Coarse Composition Framework for Knowledge Base Question Answering

  Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering

  Knowledge Graph-augmented Language Models for Complex Question Answering

  Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

  Distribution Shifts Are Bottlenecks: Extensive Evaluation for Grounding Language Models to Knowledge Bases

  Probing Structured Semantics Understanding and Generation of Language Models via Question Answering

  Keqing: Knowledge-based Question Answering is A Nature Chain-of-Thought mentor of LLMs

  Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

• Knowledge-augmented Open-domain Question Answering

  UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering

  KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

  Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering

  Grape: Knowledge Graph Enhanced Passage Reader for Open-domain Question Answering

  Enhancing Multi-modal Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation

  DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text

  KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases

  Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering

  Two-stage Generative Question Answering on Temporal Knowledge Graph Using Large Language Models

  KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph

• Table Question Answering

  NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned

  Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering

  End-to-End Table Question Answering via Retrieval-Augmented Generation

  OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

  Reasoning over Hybrid Chain for Table-and-Text Open Domain Question Answering

  Conversational Question Answering on Heterogeneous Sources

  Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

  StructGPT: A General Framework for Large Language Model to Reason over Structured Data

  cTBLS: Augmenting Large Language Models with Conversational Tables

  RINK: Reader-Inherited Evidence Reranker for Table-and-Text Open Domain Question Answering

  Localize, Retrieve and Fuse: A Generalized Framework for Free-Form Question Answering over Tables

  Exploring the Impact of Table-to-Text Methods on Augmenting LLM-based Question Answering with Domain Hybrid Data

• Others

  Improving Knowledge-Aware Dialogue Response Generation by Using Human-Written Prototype Dialogues

  Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generation

  RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding

  Retrieval-Enhanced Generative Model for Large-Scale Knowledge Graph Completion

  Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestion

  G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering

RAG for Science

• Drug Discovery

  Retrieval-based controllable molecule generation

  Prompt-based 3d molecular diffusion models for structure-based drug design

• Medical Applications

  PoET: A generative model of protein families as sequences-of-sequences

  Retrieval-augmented large language models for adolescent idiopathic scoliosis patients in shared decision-making

  BioReader: a Retrieval-Enhanced Text-to-Text Transformer for Biomedical Literature

  Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation

  From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process

• Math Applications

  Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference

  LeanDojo: Theorem Proving with Retrieval-Augmented Language Models

Benchmark

Benchmarking Large Language Models in Retrieval-Augmented Generation

CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models

ARES: An Automated Evaluation Framework for Retrieval-AugmentedGeneration Systems

RAGAS: Automated Evaluation of Retrieval Augmented Generation

KILT: a Benchmark for Knowledge Intensive Language Tasks

Citation

if you find this work useful, please cite our paper:

@article{zhao2024retrieval,
  title={Retrieval-Augmented Generation for AI-Generated Content: A Survey},
  author={Zhao, Penghao and Zhang, Hailin and Yu, Qinhan and Wang, Zhengren and Geng, Yunteng and Fu, Fangcheng and Yang, Ling and Zhang, Wentao and Cui, Bin},
  journal={arXiv preprint arXiv:2402.19473},
  year={2024}
}