Algorithm Alley
What does the term "overfitting" mean in the context of machine learning?
a) The model performs well on training data but poorly on new, unseen data.
b) The model generalizes well to new, unseen data.
c) The model is too simple to capture the underlying patterns in the data.
d) The model requires more training examples to improve performance.
a) The model performs well on training data but poorly on new, unseen data.
Explanation: Overfitting occurs when a model learns the training data too well, capturing noise or random fluctuations. As a result, it may perform poorly on new, unseen data.
What is also referred to as the "AI Platform” or the “Salesforce AI Platform”.
a) Einstein
b) EinsteinGPT
c) Hawking
d) Generative CRM
Correct Answer: Hawking
Salesforce introduced __________, which democratizes software engineering by helping users turn simple English prompts into executable code.
a) Einstein GPT
b) CodeGen
c) OpenAI playground
d) ProGen
b) CodeGen
An LLM is a next-word predictor. This seemingly simple ability leads to remarkable near-human-level abilities at a variety of Natural Language tasks. Which of these statements is false, with regard to this phenomenon:
a) These abilities became possible by training (to do next-word prediction) a sufficiently large model on a sufficiently large body of internet text, but exactly why this leads to near-human intelligent behavior is still poorly understood and an active area of research.
b) It was quite a surprise that these abilities emerged from mere next-word prediction.
c) One hypothesis is that training on next-word prediction (with a sufficiently large model and sufficiently large amount of data) somehow forces the LLM to develop an understanding of the world.
d) This is not surprising since it has long been established that the Attention mechanism in a Transformer is similar to how neurons work.
d) This is not surprising since it has long been established that the Attention mechanism in a Transformer is similar to how neurons work.
In the context of AI, what does the term "tokenization" refer to?
a) Breaking down a sentence into words or subword units.
b) Generating random tokens for security purposes.
c) Converting numerical values into categorical tokens.
d) Encrypting text data for secure transmission.
a) Breaking down a sentence into words or subword units.
Explanation: Tokenization is the process of breaking down a text into smaller units, such as words or subword units, to facilitate analysis in natural language processing tasks.
In the context of machine learning, what is a "confusion matrix" used for?
a) To measure the time complexity of an algorithm.
b) To evaluate the performance of a classification model.
c) To represent the structure of a neural network.
d) To visualize the distribution of data in a dataset.
b) To evaluate the performance of a classification model.
Explanation: A confusion matrix is a table used to evaluate the performance of a classification algorithm. It shows the number of true positives, true negatives, false positives, and false negatives.
What allows you to work with customer data to prototype new AI features and debug data issues?
a) Hawking Notebooking
b) Einstein Orchestration
c) Einstein Compute
d) Hawking Q3 Tooling
a) Hawking Notebooking
Einstein Orchestration: Orchestration enables authoring and managing ML flows.
Einstein Compute: Compute provides job control APIs, which allow you to submit and delete jobs, get job results, and subscribe to events.
Hawking Q3 Tooling: Einstein Quality Engineering (Q3) provides tools for three types of tests: end-to-end, user-scenario, and integration tests.
Salesforce AI Research trained CodeGen on a large corpus of natural and programming languages. It is an auto-regressive language model with a total of how many parameters?
a) 30 billion
b) 16 billion
c) 8 billion
d) 20 billion
b) 16 billion
A "Raw" LLM does not maintain any conversation state. But obviously on the ChatGPT website, you can ask follow-up questions, and the LLM responds by taking into account the previous dialog.
How do you think this works?
a) Whenever it is the LLM's turn to respond, the entire conversation history is sent to the LLM, so it has the full context enabling it to respond.
b) The ChatGPT web server stores conversations from all users in a database, and this powers a special module that guesses that the user most likely is asking about a topic.
c) The LLM powering ChatGPT is trained in a special way that enables it to track conversation history
d) The LLM itself maintains conversation history
a) Whenever it is the LLM's turn to respond, the entire conversation history is sent to the LLM, so it has the full context enabling it respond.
Which of the following options does not contribute to the reduction of hallucinations?
a) Use a trusted LLM to help reduce generative AI hallucinations
b) Write more-specific AI prompts
c) Incorporate a human-in-the-loop mechanism to mitigate hallucinations
d) Tell the LLM to be honest
c) Incorporate a human-in-the-loop mechanism to mitigate hallucinations
Which algorithm is commonly used for handling imbalanced datasets and is sensitive to the distribution of classes?
a) Decision Trees
b) Logistic Regression
c) Random Forest
d) K-Nearest Neighbors (KNN)
c) Random Forest constructs a multitude of decision trees during training and utilizes a bag-of-words approach to represent text data, aggregating predictions from these trees to output the mode of classes for classification or the mean prediction for regression. This methodology excels in reducing overfitting and enhancing accuracy, particularly in natural language processing tasks.
What is a search backend deployed in Hawking FDs?
a) ML Console
b) Hawking IGES (Inference Graph Execution Service)
c) VSaaS (Vector Search as a Service)
d) Feature Store
c) VSaaS (Vector Search as a Service)
Feature stores are central hubs for the data processes that power operational ML models. They transform raw data into feature values, store the values, and serve them for model training and online predictions.
ML Console provides observability and debugging functionality for ML app flows.
Hawking IGES (Inference Graph Execution Service) allows AI app dev teams to implement complex use cases that involve scoring (inferencing), multiple steps, and multiple services.
Salesforce introduced ___________ an open-source deep learning library for language-vision research. It is the most comprehensive language-vision library available today. It supports a growing list of more than 10 common language-vision tasks, across over 20 public datasets.
a) TorchMultimodal
b) UniLM
c) X-Modaler
d) LAVIS
LAVIS (short for LAnguage-VISion)
The goals of LAVIS include:
In the context of LLMs, what does "fine-tuning" refer to?
a) Adjusting hyper parameters to improve model performance.
b) Modifying the model architecture to handle larger datasets.
c) Training a pre-trained model on a specific task or domain.
d) Incorporating additional layers for better feature extraction.
c) Training a pre-trained model on a specific task or domain.
Explanation: Fine-tuning involves taking a pre-trained model (often on a large dataset) and further training it on a smaller, task-specific dataset to adapt the model to a particular domain or task.
What is the primary purpose of a TF-IDF (Term Frequency-Inverse Document Frequency) representation in NLP?
a) To measure the frequency of words in a document.
b) To identify the most important words in a collection of documents.
c) To quantify the importance of words in a document relative to a corpus.
d) To transform text into a binary format for efficient storage.
c) To quantify the importance of words in a document relative to a corpus.
Explanation: TF-IDF is used to quantify the importance of words in a document relative to a corpus by considering both the frequency of a term in a document and its rarity in the entire corpus.
What is the main idea behind transfer learning in machine learning, and how is it beneficial?
a) Utilizing knowledge gained from one task to improve learning on a different, but related task.
b) Transferring data between different models for faster convergence.
c) Sharing computational resources between multiple learning algorithms.
d) Combining the predictions of multiple models for improved accuracy.
a) Utilizing knowledge gained from one task to improve learning on a different, but related task.
Explanation: Transfer learning involves using knowledge gained while solving one problem to improve the performance on a different but related task. It's especially useful when labeled data for the target task is limited.
Data Science and Data Engineering teams are turning towards __________ to manage the data sets and data pipelines needed to productionize their ML applications. It is also called "The interface between models and data". Fill in the blanks.
a) Inference Pipeline
b) Model Store
c) Feature Generation
d) Feature Store
d) Feature Store
The feature store acts as a centralized repository for storing and managing pre-computed features
Inference Pipeline: Once a trained model is deployed, it needs to be integrated into an inference pipeline to make real-time predictions on new data.
Feature Generation: Once the data is available, the next step is to derive meaningful insights by generating relevant features - building blocks of every AI application
Model Store: After the training job is completed, the resulting model needs to be stored and versioned for future use.
What compute hardware did Salesforce use to train CodeGen?
a) Nvidia’s Titan X - Graphics processing unit(GPU)
b) Google’s v4 - Tensor processing units(TPU)
c) Xilinx’s Versal ACAP - Field-programmable gate arrays (FPGAs)
d) NVIDIA A100 Tensor Core GPU
b) Google’s v4 - Tensor processing units(TPU)
Salesforce enjoyed a close collaboration with Google on the TPU platform, a special piece of ASIC hardware devised specifically for neural network machine learning.
What is the main advantage of using transformer architectures, such as those in LLMs, for natural language processing tasks?
a) Improved sequential processing.
b) Enhanced parallelization of computations.
c) Reduced model complexity.
d) Better handling of image data.
b) Enhanced parallelization of computations.
Explanation: Transformer architectures allow for parallelization of computations, making them more efficient for processing sequences compared to traditional sequential models.
What role does the hyperparameter "learning rate" play in the training of machine learning models?
a) Controlling the speed at which the model learns from the training data.
b) Determining the size of the model's input features.
c) Specifying the number of hidden layers in a neural network.
d) Defining the threshold for classifying data points.
a) Controlling the speed at which the model learns from the training data.
Explanation: The learning rate is a hyperparameter that controls the step size during optimization, influencing how quickly or slowly a model learns from the training data.
What does the term "bias-variance tradeoff" refer to in machine learning?
a) Balancing the number of positive and negative examples in a dataset.
b) Balancing the complexity of a model with its ability to generalize.
c) Adjusting the learning rate during training.
d) Minimizing the number of features in the input data.
b) Balancing the complexity of a model with its ability to generalize.
Explanation: The bias-variance tradeoff involves finding the right level of model complexity that balances fitting the training data well without overfitting and failing to generalize.
While you are developing an AI app, you typically want to explore a customer's data to help with your app's design. As a PM which of the following is not a pre requisite for you for Streamlined Data Consent:
a) The customer has signed a UPRA (pilot agreement).
b) The customer’s org has the DataExploration permission enabled. Data cloud access is also supported but not required.
c) The PM has BlackTab (BT) access for the customer’s production org and a Change Case to run the BlackTab action in the customer’s production org.
d) The PM has access to Hawking's ML Lake to import the customer's data
d) The PM has access to Hawking's ML Lake to import the customer's data
PMs are required to run "Einstein Data Exploration Application Consent" Black Tab Action in the customer org.
What is Salesforce’s ProGen?
a) Gives Non-Coders the ability to generate code using basic text description along with testing capabilities.
b) Gives researchers great AI video platform or tool for creating videos. With little to no work, it rapidly generates and broadcasts videos of professional quality.
c) Gives scientists and researchers the ability to use AI as a tool to design highly-tailored proteins with desired properties.
d) Gives creators the ability to produce voice-overs, videos, social media postings, and logos.
c) Gives scientists and researchers the ability to use AI as a tool to design highly-tailored proteins with desired properties.
Salesforce AI Research team is currently leveraging ProGen to identify potential treatments for neurological and autoimmune disorders such as rheumatoid arthritis and multiple sclerosis. Salesforce’s ProGen project revealed that by creating language models based around amino acids instead of letters and words, generative AI was able to produce proteins that have not been found in nature, and in many cases, are more functional.
What is the primary function of attention mechanisms in Large Language Models (LLMs) like GPT-3?
a) To control the flow of information within the model.
b) To increase the model's memory capacity.
c) To speed up training time.
d) To optimize model architecture.
a) To control the flow of information within the model.
Explanation: Attention mechanisms in LLMs allow the model to focus on different parts of the input sequence when generating each part of the output sequence, enabling more effective language understanding and generation.
What is the primary goal of ensemble learning in machine learning?
a) To reduce model complexity for faster training.
b) To combine predictions from multiple models for improved accuracy.
c) To prevent underfitting by increasing the number of training examples.
d) To enforce diversity in the feature space.
b) To combine predictions from multiple models for improved accuracy.
Explanation: Ensemble learning involves combining the predictions of multiple models to achieve better overall performance than individual models, often through techniques like bagging or boosting.