Robo-Decisionland (Decision-Making)
Define 'policy' in the context of ML.
Kudos Point: what is the difference between deterministic & stochastic policies?
A policy is a set of rules or strategies that govern the decision-making process of an autonomous agent. They are most associated with RL, which is a type of learning that occurs when an agent learns to make decisions by interacting with an environment (repeatedly).
Deterministic policy refers to policies that have NO randomness involved - when an agent is in a specific state, it will always choose a specific action. (1-to-1 mapping)
Stochastic policy refers to policies in which the agent's actions are probabilistic, which means that when an agent is in a specific state, there exists a probability distribution over several actions that may lead to several states. This adds randomness in the decision-making process.
Which NLP task involves determining the sentiment expressed in a piece of text, such as positive, negative, or neutral?
Kudos Point: What is a military application of this task? What is a non-military application of this task?
Sentiment Analysis
Threat detection is one application - monitoring online forums and platforms to detect potential threats or hostile activities. Others include Psychological Operations (the psychological impact of military actions on local populations), information warfare (misinformation or propaganda helps the military identify and counter disinformation campaigns), etc
Other applications can include political sentiment (public opinions about political figures and policies), music sentiment (understand emotional tone of song lyrics for music recommendation systems), and social media influence!
Name a common, popular algorithm for finding a path on a graph.
breadth first search (BFS), depth first search (DFS), etc.
What separates a decentralized policy from a centralized policy in the context of multi agent autonomy?
In a centralized policy, one agent/computer is making all the decisions for the team in a coordinated or collaborative fashion. In a de-centralized policy, each agent is making their own decision(s) independently. Agents may still choose to coordinate or collaborate in a decentralized environment.
What is the name of the android that serves on the Enterprise on Star Trek: The Next Generation?
Data
Define Imitation Learning.
Kudos Point: Provide an example of an algorithm using imitation learning.
Imitation learning is a type of Reinforcement Learning in which the agent attempts to improve its policy by observing, mimicking, and/or learning from a teacher.
Examples of Imitation Learning include:
- Behavioral Cloning: we want to learn the teacher's policy using supervised learning
- Interactive Imitation Learning: we want to create a policy through observing the expert & querying them on "what would you have done"? (iterative style)
- Inverse RL: we want to surpass the expert by learning the reward function from their demonstrations, and subsequently learning our own policy that will perform BETTER than theirs.
What computer vision task involves identifying and labelling objects within an image?
Kudos Point: Why is this a difficult task?
Object Recognition or Object Detection
We don't have a comprehensive database of all objects in an image, so much of our existing corpus is human-labeled data! Which is expensive!
Name a common, popular algorithm for finding the shortest path on a graph.
Kudos: Explain why it will return the shortest path.
A* algorithm returns the shortest, most optimal path on a graph as it considers both the cost to reach a node & an estimate (via a heuristic) of the remaining cost to reach the goal.
0
1 2
3 4 5 6
Here is an example of a tree. Which approach explores - in order - nodes 0, 1, and 3 first?
Depth first search
What is the name of the AI that won the 2011 Jeopardy! episode against famed contestant & current Jeopardy! host, Ken Jennings?
IBM Watson
Define S.A.T.R. in the context of Markov Decision Processes (MDPs)
S is the set of States that the robot can reach
A is the set of Actions that the robot can take
T is the Transition Function (probability of the next state s' given that your robot is in state s and takes action a)
R is the Reward Function
What is the purpose of stemming in NLP?
To reduce words to their root or base form, which is often used to improve text analysis or information retrieval.
What does the term 'evolutionary algorithm' refer to in the context of AI?
Kudos Point: Name a few components of evolutionary algorithms
Evolutionary algorithms are a broad category of optimization solutions inspired by the process of natural selection and evolution, and a popular subfield within the bioinformatics research community. It simulates the process of evolution to find solutions to complex problems. Key components to evolutionary algorithms include:
1. Initialization - a population of potential solutions is initialized
2. Evaluation - each individual in the population is evaluated via a fitness function
3. Selection - individuals are chosen for reproduction based on their fitness.
4. Crossover/Recombination - genetic material from selected individuals are combined to create offspring.
5. Mutation - random changes are introduced to maintain diversity
6. Replacement - new individuals replace the old generation
7. Termination - the algorithm ceases after a termination condition is met or a certain number of generations have passed.
What does the algorithm SLAM stand for?
Kudos Point: Can you describe the algorithm? In what context is it used?
Simultaneous Localization And Mapping
This is a technique used by an agent to create a map of an unknown environment while also self-localizing the robot within the robot (aka keeping track of its location).
In SLAM, the robot uses sensors - LIDAR, radar, visual, etc - to gather information about the environment. As it moves, the robot continuously estimates its own position within the map it is building. This is useful in environments where the robot doesn't have prior knowledge of the environment.
What is the name of the Pixar animated film featuring a trash-compacting robot as the main character?
WALL-E
In the context of Q-MIX, what does the acronym CTDE stand for?
Kudos Point: When would we need to use CTDE?
Centralized Training and Decentralized Execution
You may use CTDE in scenarios where agent-to-agent communication is limited, such as in comms-denied or suppressed environments, but necessitates collaboration (and can benefit from communicating during training).
Which computer vision application involves determining the 3D structure of objects from 2D images?
Stereo Vision or Depth Perception
What is an algorithm that describes this behavior? Hint: The context is dynamic programming
Input 1: arr = {3, 10, 2, 1, 20}
Output 1: 3
Explanation 1: The _____ is 3, 10, 20
Input 2: arr = {3, 2}
Output 2: 1
Explanation 2: The _____ are {3} and {2}
Input 3: arr = {50, 3,10, 7, 40, 80}
Output 3: 5
Explanation 3: The ______ is {3, 7, 40, 80}
This is the longest increasing subsequence (LIS)
This is the longest possible subsequence in which the elements of the subsequence are sorted in increasing order.
A subsequence is a new string generated from the original string with some characteristics (but may be none) deleted, but the original order of the original string must be preserved.
A substring is a contiguous, non-edited sequence of characters of an original string. In Example 3, the longest common substring would be {7, 40, 80}
What do the acronyms LIDAR and RADAR stand for?
LIDAR stands for LIght Detection and Ranging
RADAR stands for Radio Detection and Ranging
What is the name of Tony Stark's AI-powered personal assistant in the 2008 film "Iron Man"?
J.A.R.V.I.S. (Just A Rather Very Intelligent System)
What is submodular maximization?
Kudos Point: Can you provide an example?
Combinatorial optimization problems where the goal is to find a subset of items from a larger set in a way that a specific submodular function is maximized over that subset. Often, they have diminishing returns, meaning that adding items to a larger set has a smaller 'reward' than adding items to a smaller set.
Example: the social media & advertising paradigm. If you want to find a subset of social media influencers that can advertise your product, you want to find the most diverse set of followers to maximize coverage/exposure.
Which computer vision task involves recognizing and understanding the content of an entire scene or image?
Kudos Point: Why, from a military perspective, do we care about this task?
Scene Understanding or Image Understanding
The goal is to extract high-level information about the relationships between their objects, their spatial arrangement, and the overall context of the scene.
In graph theory, what does a 'cycle' represent?
Kudos Point: Can you have a cycle in a directed graph? Can you have a cycle in an undirected graph?
Kudos Point 2: Are edges distinct?
Kudos Point 3: Are vertices repeated?
Kudos Point 4: What is a graph called that does NOT have cycles?
A cycle is a path in a graph that starts and ends at the same vertex, traversing a sequence of edges such that no vertex (except for the start/end vertex) is repeated. It is a closed loop within the graph.
You can have a cycle in both a directed and undirected graph. Edges ARE distinct. A graph without cycles is called 'acyclic'.
Which swarm behavior involves robots aligning their movements with the average direction of their neighbors?
Flocking behavior
What is the name of the 2014 film featuring a robot named Baymax who provides healthcare assistant to the protagonist?
Big Hero 6