Architecture
An animal famous for its proximity to the prevailing architecture of today’s NLP research.
What is Llama?
Pan-Continental grassroots organization representing NLP research communities from large landmass whose population is separated by a large desert.
What is Masakhane?
Teaching a model that one option is better than the other
What is Direct Preference Optimization/DPO?
To test how well a model can understand and provide a solution to programming issues specific to an online repository.
What is SWEBench?
Keeps saying AI has hit a wall
Who is Gary Marcus?
Possibly a woman’s name with a single lowercase letter. For the GPU-Poor.
What is LoRA?
The organization attributed to the creation of a dataset named after what you get after stacking many things of similar but varying properties.
What is EleutherAI?
We (LTI community) can’t live without it but it keeps going down every now and then.
What is Babel?
A staple to pretraining that’s really just the open internet
What is Common Crawl?
Claimed 9 years ago that a medical profession would be gone 4 years ago because of deep learning.
Who is Geoff Hinton?
Model component often mistaken as N-models in a trenchcoat
What are Mixture-of-Experts?
Site named after a situation that is better than being wrong but not quite right
What is Less Wrong?
Not in the beginning but also not the end of training
What is Mid Training?
Data derived from the very thing we sought to train
What is Synthetic Data?
For the concept of training a model by telling it "Bad" vs "Good", Winner of the award named after the person who argued intelligence could be tested by whether a human subject could not accurately distinguish between computer or human interlocutors.
Who are Andrew Barto and Richard Sutton?
Component invented by a researcher whose last name can also be read as an English Pronoun
What are Residual Connections?
An organization that, about 9 years ago, launched a “Conversational Understanding” bot online that quickly learned to hurl slurs and insults.
What is Microsoft?
Imagines as the top of a black forest cake which a certain NYU professor noted as part of a model’s training
What is Reinforcement Learning/RL?
Multilingual, Portuguese Named, also name of an Island in Indonesia
What is FLORES?
Athlete, winning 1 of 5 matches in a historic game of man vs machine that would convince a company of buying another company.
Who is Lee Sedol?
The first word of a foundational paper also noted for having authors whom are no longer affiliated with the organization that published it.
What is Attention?
Non-Profit, An Institution Associated with Intelligence
What is AI2?
Reward Increment = Nonnegative Factor x Offset Reinforcement x Characteristic Eligibility
What is REINFORCE?
An open dataset from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. Made by folks on the west coast.
What is Dolma?
"I tend to think that most fears about A.I. are best understood as fears about capitalism"
Who is Ted Chiang?