ML Foundations
Name the three primary learning paradigms in Machine Learning.
Supervised learning, Unsupervised learning, and Reinforcement learning.
Expand the abbreviation 'PAC' in PAC learning.
Probably Approximately Correct learning.
Expand the abbreviation 'PAC' in PAC learning.
Probably Approximately Correct learning.
Define 'version space' in concept learning.
The version space is the subset of all hypotheses in the hypothesis space that are consistent with all the training examples observed so far.
Name two real-world domains where Machine Learning is widely applied.
Any two of: healthcare (diagnosis), finance (fraud detection), e-commerce (recommendations), autonomous vehicles, speech/image recognition, etc.
What is a 'hypothesis' in the context of a learning algorithm?
A hypothesis is a candidate function or model, mapping inputs to outputs, that the learning algorithm proposes as an approximation of the true underlying target function.
Differentiate Supervised and Unsupervised learning, with one example of each.
Supervised learning uses labelled input-output pairs to learn a mapping (e.g. predicting house prices from features). Unsupervised learning works with unlabelled data to find structure or patterns (e.g. customer segmentation via clustering).
In PAC learning, what do the parameters epsilon (ε) and delta (δ) represent?
Epsilon (ε) is the accuracy parameter — the maximum allowed error between the learned hypothesis and the true concept. Delta (δ) is the confidence parameter — the maximum allowed probability that the learner fails to achieve that accuracy.
What do the General boundary (G) and Specific boundary (S) represent in a version space?
S is the set of most specific hypotheses consistent with the data (covering only positive examples seen), and G is the set of most general hypotheses consistent with the data (excluding all negative examples seen); together they compactly bound the entire version space.
How does Machine Learning enable AI systems used in speech recognition applications?
ML models are trained on large datasets of spoken audio paired with transcriptions, learning statistical patterns that map acoustic signals to phonemes/words, allowing the system to generalise to new speakers and accents rather than relying on fixed acoustic rules.
What is a 'hypothesis space' and how does it differ from a single hypothesis?
The hypothesis space is the entire set of all candidate hypotheses (functions/models) that a learning algorithm is allowed to choose from, defined by the chosen model class; a hypothesis is one specific member of that space.
What is Reinforcement Learning, and give one real-world example.
Reinforcement Learning is a paradigm where an agent learns to take actions in an environment to maximise cumulative reward through trial and error and feedback. Example: a robot or game-playing agent (e.g. AlphaGo) learning strategy through rewards/penalties.
What does it mean for a concept class to be 'PAC learnable'?
A concept class is PAC learnable if there exists a learning algorithm that, for any target concept in the class, any distribution over inputs, and any ε, δ > 0, can output a hypothesis that is approximately correct (error ≤ ε) with high probability (≥ 1−δ), using a number of samples and computation time polynomial in 1/ε, 1/δ, and other problem parameters.
Briefly describe how the Candidate Elimination algorithm updates S and G as new training examples arrive.
For a positive example, hypotheses in G inconsistent with it are removed, and hypotheses in S are generalised minimally to cover it. For a negative example, hypotheses in S inconsistent with it are removed, and hypotheses in G are specialised minimally to exclude it, after which inconsistent or non-boundary hypotheses are pruned.
Explain how Machine Learning powers recommendation systems (e.g. on e-commerce or streaming platforms).
Recommendation systems use ML techniques (e.g. collaborative filtering, content-based methods) to learn patterns from users' past behaviour and preferences (ratings, clicks, purchases) and predict items a user is likely to want next, personalising suggestions without explicit rule-writing.
Explain why Machine Learning is considered essential to enabling AI applications rather than just a nice-to-have.
Many AI applications (vision, speech, decision-making) involve tasks too complex for hand-coded rules; ML allows systems to automatically discover patterns from data and improve with experience, making adaptive, scalable intelligent behaviour possible.
What is Semi-supervised learning, and when is it useful?
Semi-supervised learning uses a small amount of labelled data together with a large amount of unlabelled data to build a model. It is useful when labelling data is expensive/time-consuming but unlabelled data is abundant, e.g. medical image diagnosis.
Explain the relationship between sample complexity and the PAC learning guarantee.
Sample complexity is the number of training examples required to guarantee, with probability at least 1−δ, that the learned hypothesis has error at most ε. A PAC guarantee is only meaningful if this sample complexity is finite and grows polynomially (not exponentially) with 1/ε, 1/δ, and the complexity of the hypothesis space.
What is the effect on the version space when a new positive example is seen versus when a new negative example is seen?
A new positive example removes overly specific hypotheses from S (generalising S) and eliminates hypotheses in G that fail to cover it; a new negative example removes overly general hypotheses from G (specialising G) and eliminates hypotheses in S that incorrectly cover it. In both cases the version space shrinks or stays the same, never grows.
Discuss the role of Machine Learning in the perception system of a self-driving car.
ML models (e.g. for object detection, classification, segmentation) process sensor data (camera, LiDAR, radar) to identify and track objects like pedestrians, vehicles, and lane markings in real time, enabling the car to build an understanding of its environment for safe navigation and decision-making.
Contrast the traditional rule-based programming approach with the Machine Learning approach, using an example such as spam detection.
In rule-based programming, a human explicitly writes fixed rules (e.g. 'block email if it contains word X') that must be manually updated. In ML, the system is given labelled examples of spam/non-spam and automatically learns the rules/patterns from data, generalising to new, unseen emails and adapting as spam tactics change.
A company has a small set of confirmed fraud cases and a huge pool of unlabelled transactions, and wants to detect fraud. Which learning paradigm is most suitable, and justify your answer.
Semi-supervised learning is most suitable, since it can leverage the few labelled fraud/non-fraud examples along with the large unlabelled transaction pool to build a more accurate and generalisable fraud-detection model than using the small labelled set alone.
Conceptually, how does required sample size behave as epsilon decreases, as delta decreases, and as the hypothesis space H grows larger? Explain the intuition.
Sample size increases as ε decreases (higher accuracy needs more data), increases as δ decreases (higher confidence needs more data), and increases as |H| grows (a larger hypothesis space is harder to narrow down reliably, roughly following bounds like O((1/ε)(ln|H| + ln(1/δ)))). Intuitively, tighter accuracy/confidence requirements and richer model classes all demand more evidence.
Under what condition does the version space converge to a single hypothesis, and what happens if no hypothesis in H is consistent with all training data?
Version space converges when one hypothesis matches all training examples. If none matches, the version space becomes empty (∅).
Argue why Machine Learning is essential — not optional — for achieving true AI capability, citing a specific application domain to support your argument.
True AI requires handling the enormous variability and complexity of real-world data (e.g. countless visual scenes in autonomous driving), which cannot be captured by exhaustive hand-written rules; ML allows systems to automatically learn generalisable patterns from data and continually improve, making adaptable, scalable intelligence achievable in domains like autonomous driving where explicit rule-based programming would be infeasible.