Algo SBS
What are we judging in Algo SBS?
What are the 3 Intents?
1. major intent (most common)
2. minor intent (less popular)
3. rare intent (not very popular)
What summarizes the “SuperFresh” Event after reading the query?
The Query Interpretation.
When judging Page
Usefulness, we should
judge main content only... True or false?
True
What are we judging here?
Two sets of Video sides
Query: Fish
Right side shows good quality images of fish, no issues, left side shows good quality images with no issues. What's the rating?
Neutral
Query: "Church Door". Video is showing the entry of a church.
We would avoid using extreme ratings, such as “Highly Relevant” or “Not Related”. True or False?
True!
What are the three timings of an event?
Pre-Event, During Event, After the Event.
What intents are there in MMPU?
Image Intent - Informational Intent - Mixed intent
What is ‘metadata’
The text information for each video. (Title, caption, author etc)
One side is blank the other is irrelevant, what's the rating?
Blank side wins
If the website is talking about irrelevant information to the video that is displayed on the website, we downgrade by -1. True or false?
videos sometimes appear in webpages that are not relevant to the video. And that's okay because we only judge the Video not the page.
Judge the image in the collage that is “nearest” to the "SuperFresh" event to determine what instance of the event this image represents. Is that True or False?
True!
.If the page contains highly specialized
information, you can either select “cannot judge” or
keep on judging depending on how familiar you are
with the topic. True or False?
True
Videos are missing important meta deta, what's the rating?
MISSING METADATA
Both sides are equally irrelevant - what's the rating?
Neutral
Video matches the intended subject; however, it contains additional, less-useful content moves away from the user’s intent.
What rating is that?
Relevant.
What are the 5 Instance Ratings for SFMMIR?
Yes, Image is of Specific Event in Interpretation, Generic Image, No, Main Character Out of Context, No, Incorrect Instance of Event, Unable to Determine / Can’t Judge.
Page link redirects to a different page - whats the rating
broken
The query "Cost of living inflation 2023" returns five video results on the left-side and two video results on the right-side, we judge it as can't judge because the amount of videos on each side are uneven - true or false?
False - Do not judge based on number of results, judge based on the relevance of results.
One side is blank, the other didnt load but displays images icons. What's the rating?
Didn't load.
Query: Robin Williams Interview
Market Language: Spanish
Video Language: English
What's the language rating?
Does not degrade. It's expected for Robin Williams to speak English.
Query is Indiana Jones. Query Interpretation: On June 30th, the new and last Indiana Jones movie was released in the Movies. The user would like to see images of the movie.
The image is a Movie Poster of the new Indiana Jones. What's the rating?
Relevant - Yes correct instance - during the event
Some pages will be outdated or “stale”. Consider the trendiness or newness of a page when making your judgement. If the page is so old that it will not satisfy a user with recent information. What do we do?
downgrade by one
For channel queries, you will most often prioritize the side with the freshest results as the winning side. True or false?
True