General LLM Stuff
OpenAI's most popular LLM
ChatGPT
What is the first step of pretraining?
Web scraping
SFT stands for
Supervised Fine Tuning
RLHF stands for
Reinforcement Learning from Human Feedback
What do AI Agents do?
Splits up tasks into multiple steps
What is a large language model?
A neural network that processes and generates human-like text
One factor that allows an LLM to be "bigger" than another
More parameters
More tokens
Bigger context window
Is SFT done by humans or fully AI?
Humans
How does Reinforcement Learning work?
Prompt LLM many times, incentivize the correct answers
Name an example of an AI Agent
Coding, Web Search, Weather, etc.
The difference between multimodal and LLMs
LLMs only handle text, multimodals can take on images, audio, etc.
What is an inference?
A forward pass through the model
Two ways to fix hallucinations
Train it to say "I don't know"
Usage of tools
Difference between RL and RLHF
RL is used for concrete answers, RLHF for indefinite ones, with human assistance
Give an example of what Agents can do with a computer
They can perform multiple operations, browse, test out code, debug, etc.
What architecture is most commonly used in LLMs?
Transformers
Tokenization uses ____-____ encoding
byte pair
What is the point of <User> and <Assistant> tokens?
To provide the LLM knowledge of the user's prompt and where it should begin its response
What happens if you run RLHF too many times?
Finds a loophole to the model with nonsensical responses
Why do reasoning models take so much longer?
They "think," taking different approaches and checking work before providing a solution
Why can't LLMs count/spell?
Don't see one by one, it's just a token generator
What is a base model, and why is it different than an LLM that we use?
Base models are just token generators, not fine tuned to produce answers to questions or keep a conversation
Give an example of a hallucination and why it would happen
Many possible answers
How does RLHF avoid humans having to rate a billion LLM outputs?
Uses another neural network that simulates human scoring
Why are AI Agents regarded as the future?
Not just prompts and responses, they can get complex multi-step jobs done, something AI is yet to do