Fake it 'til You Retrieve It!
RAG was proposed partly to address this common LLM habit of stating made-up facts with the confidence of a man explaining wine.
What is hallucination?
These are the two RAG model variations proposed in the original Lewis et al. paper, differing in whether retrieved documents are fixed across or varied within a generated sequence.
What are Sequence and Token?
Featured in the original RAG paper, this benchmark’s name makes it sound like the world’s calmest interrogation.
What is Natural Questions?
One big implication of RAG is that model knowledge becomes easier to update, because you can revise the retriever’s corpus instead of doing this expensive thing again.
What is retraining the model?
Unlike standard parametric models, RAG can consult this kind of memory at inference time instead of squeezing every fact into frozen weights.
What is external knowledge?
This collaborative online encyclopedia, containing over 21 million documents, serves as the knowledge index in the RAG paper's retrieval system.
What is Wikipedia?
This benchmark, also used in the RAG paper, sounds less like quality assurance and more like a meta pass at our current activity.
What is TriviaQA?
This key advantage of RAG allows models to update their knowledge about the world without retraining, simply by modifying an external component.
What is updating knowledge by replacing the external document index?
A major motivation for retrieval-based generation is that knowledge can be updated by changing this, without retraining the whole model.
What is the document index?
Developed by Facebook AI Research, this efficient similarity search library is used in the RAG paper to index and retrieve dense document embeddings at scale.
What is FAISS?
In RAG-style retrieval systems, one common retrieval metric checks whether the correct supporting document appears within the top [blank] results?
What is k?
In RAG, errors increasingly stem from this component, as incorrect or irrelevant outputs are often caused not by generation itself, but by failures in selecting appropriate supporting information.
What is the retrieval component (or retrieval quality)?
RAG helps on knowledge-intensive NLP tasks because it combines generation with access to evidence relevant to this.
What is the input query?
This large pre-trained encoder-decoder model from Facebook serves as the generator component in the RAG framework, producing final text outputs conditioned on retrieved documents.
What is BART-large?
One reason RAG often beats purely parametric baselines on knowledge-heavy tasks is that the model can ground answers in these retrieved items.
In the RAG framework, this internal component provides fluency and implicit knowledge, while the external memory provides explicit and updatable information.
What is parametric memory (the generator / model weights)?